commlib

The specification, server implementation, and client implementation.

Actors

• Collector Service - Software that aggregates and monitors the status of the collection.
• FSA - Full Screen Application. Runs on the head unit and is what the pilot sees.
• LOB - Line of Business Application, aka Magic Carpet. Also contains an interface for OCC to interact with the collector.
• OCC - Operations Control Center. People that are responsible for monitoring the minute by minute operations of the aircraft and assisting the collector Operator with anything they can.

Supported Ciphers (Cipher is to TSKDF)

AES_256_CTR_TSKDF_HMAC_SHA2_512

AES_256_CTR_TSKDF_HMAC_SHA3_512

AES-256 in CTR mode using an Initialization Vector (IV) (also called a Nonce in CTR Mode) and key generated with a Time and Sequence Based Key Derivation Function (TSKDF).
The TSKDF uses the Hash indicated in the cipher name.
The TSKDF is used to protect the messages from birthday attacks on the sequence counter since it is likely to be limited to ~16-bits. That is, to avoid any issue with the sequence number reuse because we go through a large number of sequence numbers in a given session and we are starting from an aribtrary position of a number that otherwise only needs to (each about 60000[1] to do its job of protecting the stream from the transport issues of the Internet such as misrouting or getting stuck in buffers.
[1] 1000 messages a second * 60 seconds in 1 minute = 6000. If we are able to reach a cadence of 1000 messages per second we are also by definition not on a network that misrouting and arbitray delays of messages will happen. See Command And Control protocol description for more information on the cadence of the messages.
We include both the time window and the sequence number for the IV just remove any question about the reuse of IVs (which would otherwise occur regularly)
We include both the time window and the sequence number for the key because we can and it does not make any part of the implementation harder.
We need to change either the keys and/or the IV with every message because these messages will be very repetative and very low entropy on its own.
The use of a derived key as the key that is used for all further derivations is to allow the master key to removed from memory as soon as possible.

Support Message Authentication Codes (MAC)

HMAC_SHA2_256_KDF_HMAC_SHA2_512

HMAC_SHA3_256_KDF_HMAC_SHA3_512

A standard HMAC using the Hash and Mode indicated in the MAC name with a key derived from the master key using a KDF and utilizing the Hash and mode named in the MAC name.

Time and Sequence Based Key Derivation Function - TSKDF

Using the same principle that TOTP (Time-based One Time Pad) (RFC6238) uses to extend a HOTP (Hash-based One Time Pad), we are using a Key Derivation Function (KDF) for generating unique keys and IVs for every message and timestep so that we do not reuse the the combination of the IV, Key, and Payload.
We use this same timestep and message counter to derive a key for use in the signature algorithim as well.
X represents the time step in seconds (default value X = 30 seconds) and is a system parameter.
T0 is the Unix time to start counting time steps (default value is 0, i.e., the Unix epoch) and is a system parameter.
Timestep (T) is an integer and represents the number of time steps between the initial counter time T0 and the current Unix time. Specifically, timestep = floor((Current Unix time - T0) / X).
For example, with T0 = 0 and Time Step X = 30, T = 1 if the current Unix time is 59 seconds, and T = 2 if the current Unix time is 60 seconds.
The implementation of this algorithm MUST support a time value T larger than a 32-bit integer when it is beyond the year 2038. The value of the system parameters X and T0 are pre-established during the provisioning proces and communicated between a prover and verifier as part of the provisioning step.
That is, the iv and key are predictable if you have the key.
We end up with a hierarchy of keys as below:

graph TD;
    MK[Master Key];
    HK[HMAC Key];
    EK[Encryption Key];
    MKIV["Message Key & IV"];
    MK-- KDF -->HK;
    MK-- KDF -->EK;
    EK--TSKDF Timestep,Seq-->MKIV;

NOTE: While we give this its own name and a specific structure this is a specialization of a general solution was outlined by NIST in October 2009, in Special Publication 800-108, for this sort of use case, so rather than rolling our own algorithm, we are using a Key based Key Derivation Function that is built upon an HMAC which was constructed using SHA512. An archival copy of this publication can be found in the documentation directory of this repository.

PROTOC BASICS

Installation

https://medium.com/@erika_dike/installing-the-protobuf-compiler-on-a-mac-a0d397af46b8

Creating PROTOC Classes

https://developers.google.com/protocol-buffers/docs/pythontutorial protoc -I=/Users/nicholasho/Documents/ClearGrid/commlib --python_out=/Users/nicholasho/Documents/ClearGrid/commlib /Users/nicholasho/Documents/ClearGrid/commlib/commlib/proto/location.proto

SETUP

See settings/settings.py to set up the server settings The client and server libraries can also be invoked with arguments in the following order:

  server = UDP.SocketServerClass(
  SECRET_KEY, TSKDF_LENGTH, UDP_IP_ADDRESS, UDP_PORT_NO
  )
  client = UDP.Client(SECRET_KEY, TSKDF_LENGTH, UDP_IP_ADDRESS,
 UDP_PORT_NO)

Constructing a Message

Constructing a message requires messageconstructor.py and payloadconstructor.py
This allows both the server, and the client to construct messages easily.

1. constructed_message = construct_message(collector, sequence, cipher, mac)
2. Create a payload_message ping = PayloadConstructor.create_ping()
3. augmented_payload = augment_payload(payload, 'ping', ping)
4. data = encrypt_and_sign_message(constructed_message, augmented_payload, cipher, mac)
5. self.sock.sendall(data)

SERVER SETUP

server = UDP.SocketServerClass(
        secret_key,
        TSKDF_LENGTH,
        bind_ip,
        bind_port,
        cipher,
    )

Register any message handlers using the following:

@server.register_dispatch("prepare_mission_message")

Running Commlib

Client

Client needs to initialize client by calling client = commlib.UDP.Client() with the following kwargs

• secret_key,
• tskdf_length,
• udp_ip_address,
• udp_port_no,
• cipher,
• mc_url,
• collector_pk,
• username,
• password
  These following can be set by setting the Env Vars: UDP_IP_ADDRESS, TSKDF_LENGTH, UDP_PORT_NO, MC_URL, MC_USERNAME, MC_PASSWORD, SECRET_KEY, COLLECTOR_PK. These also have default variable settings.

Initializing Information from MC

The client attempts to acuqire information from MC through two calls:

1. Getting the mission_pks for the collector (can be more than 1 mission active, although not advised)
2. Initializing the device_sync for each mission_pk. This syncs the collector device_ids to what is reflected on the server

Server

Server needs to initialize client by calling server = commlib.UDP.SocketServerClass() with the following kwargs

• secret_key,
• tskdf_length,
• udp_ip_address,
• udp_port_no,
• cipher,
  These following can be set by setting the Env Vars: UDP_IP_ADDRESS, TSKDF_LENGTH, UDP_PORT_NO. These also have default variable settings.

Default Settings

Default settings ensure that the client and server can communicate in a local setting.

Versioning

This project follows Semantic Versioning except with how prereleases are encoded as python package indexes do not allow x.y.z-YYYY.MM.DD.HH.MM.SS. Instead we encode builds on master as beta releases and builds on every other branch as alpha releases and are encoded as x.y.zaYYYYMMDDHHMMSS

DevPi Setup

DevPi Server Setup

1. Install the Server: pip install -U devpi-server or pip install -U devpi-web (Recommend Devpi Web)
2. Start and initialize the Server: devpi-server --start --init
3. Start the server (after initialized): devpi-server --start
4. Additional configuration for indexing/permanent installation....

DevPi Client Setup

1. Install the Client pip install -U devpi-client
2. Start the Client devpi use http://localhost:3141
3. Login devpi login testuser --password=123
4. devpi use testuser/dev
5. Install Packages devpi install package

Setting Credentials after Initial Start

1. devpi login root --password ''
2. Set Root Credentials: devpi user -m root password=123
3. devpi user -c testuser password=123
4. devpi login testuser --password=123

Uploading a Package

1. Create an index that is not a mirror: devpi index -c dev bases=root/pypi
2. Choose index devpi use testuser/dev
3. NOTE: Only committed files will appear in a package!!!
4. devpi upload in same directory as setup.py
   https://devpi.net/docs/devpi/devpi/stable/+doc/quickstart-releaseprocess.html#installing-devpi-client-and-server