Metadata-Version: 2.1
Name: cgr-commlib
Version: 0.1.1b20200128183716.post2
Summary: clearGRID Collector Communcation Library
Home-page: https://gitlab.us-central-1.7627.network/cleargrid/abu/commlib
Author: ClearGrid
Author-email: packaging@cleargrid.io
License: UNKNOWN
Description: # commlib
        
        The specification, server implementation, and client implementation.
        
        # Actors
        
        - `Collector Service` - Software that aggregates and monitors
          the status of the collection.
        
        - `FSA` - Full Screen Application. Runs on the head unit and
          is what the pilot sees.
        
        - `LOB` - Line of Business Application, aka Magic Carpet.
          Also contains an interface for OCC to interact with the
          collector.
        
        - `OCC` - Operations Control Center. People that are responsible
          for monitoring the minute by minute operations of the aircraft
          and assisting the collector Operator with anything they can.
        
        # Supported Ciphers (Cipher is to TSKDF)
        
        ## AES_256_CTR_TSKDF_HMAC_SHA2_512
        
        ## AES_256_CTR_TSKDF_HMAC_SHA3_512
        
        AES-256 in CTR mode using an Initialization Vector (IV)
        (also called a Nonce in CTR Mode) and key
        generated with a Time and Sequence Based Key Derivation
        Function (TSKDF).
        
        The TSKDF uses the Hash indicated in the cipher name.
        
        The TSKDF is used to protect the messages from birthday
        attacks on the sequence counter since it is likely to be
        limited to ~16-bits. That is, to avoid any issue with
        the sequence number reuse because we go through a
        large number of sequence numbers in a given session and
        we are starting from an aribtrary position of a number that
        otherwise only needs to (each about 60000[1] to do its job
        of protecting the stream from the transport issues of the
        Internet such as misrouting or getting stuck in buffers.
        
        [1] 1000 messages a second \* 60 seconds in 1 minute = 6000.
        If we are able to reach a cadence of 1000 messages per second
        we are also by definition not on a network that misrouting and
        arbitray delays of messages will happen. See Command And Control
        protocol description for more information on the cadence of the
        messages.
        
        We include both the time window and the sequence number for
        the IV just remove any question about the reuse of IVs
        (which would otherwise occur regularly)
        
        We include both the time window and the sequence number for
        the key because we can and it does not make any part of the
        implementation harder.
        
        We need to change either the keys and/or the IV with every
        message because these messages will be very repetative and
        very low entropy on its own.
        
        The use of a derived key as the key that is used for all
        further derivations is to allow the master key to removed
        from memory as soon as possible.
        
        # Support Message Authentication Codes (MAC)
        
        ## HMAC_SHA2_256_KDF_HMAC_SHA2_512
        
        ## HMAC_SHA3_256_KDF_HMAC_SHA3_512
        
        A standard HMAC using the Hash and Mode indicated in the
        MAC name with a key derived from the master key using
        a KDF and utilizing the Hash and mode named in the MAC name.
        
        # Time and Sequence Based Key Derivation Function - TSKDF
        
        Using the same principle that TOTP (Time-based One Time Pad)
        ([RFC6238](https://tools.ietf.org/html/rfc6238)) uses to
        extend a HOTP (Hash-based One Time Pad), we are using
        a Key Derivation Function (KDF) for generating
        unique keys and IVs for every message and timestep
        so that we do not reuse the the combination of the
        IV, Key, and Payload.
        
        We use this same timestep and message counter to derive a key
        for use in the signature algorithim as well.
        
        `X` represents the time step in seconds (default value X =
        30 seconds) and is a system parameter.
        
        `T0` is the Unix time to start counting time steps (default
        value is 0, i.e., the Unix epoch) and is a system parameter.
        
        Timestep (`T`) is an integer and represents the number of
        time steps between the initial counter time T0
        and the current Unix time. Specifically,
        `timestep = floor((Current Unix time - T0) / X)`.
        
        For example, with `T0 = 0` and `Time Step X = 30`, `T = 1` if
        the current Unix time is 59 seconds, and `T = 2` if the
        current Unix time is 60 seconds.
        
        The implementation of this algorithm MUST support a time
        value T larger than a 32-bit integer when it is beyond
        the year 2038. The value of the system parameters X
        and T0 are pre-established during the provisioning proces
        and communicated between a prover and verifier as part of
        the provisioning step.
        
        That is, the iv and key are predictable if you have the key.
        
        We end up with a hierarchy of keys as below:
        
        ```mermaid
        graph TD;
            MK[Master Key];
            HK[HMAC Key];
            EK[Encryption Key];
            MKIV["Message Key & IV"];
            MK-- KDF -->HK;
            MK-- KDF -->EK;
            EK--TSKDF Timestep,Seq-->MKIV;
        ```
        
        NOTE: While we give this its own name and a specific
        structure this is a specialization of a
        general solution was outlined by NIST in October 2009, in
        Special Publication 800-108, for this sort of use case, so
        rather than rolling our own algorithm, we are using a _Key
        based Key Derivation Function_ that is built upon an HMAC which
        was constructed using SHA512. An archival copy of this
        publication can be found in the documentation directory of
        this repository.
        
        # PROTOC BASICS
        
        ## Installation
        
        https://medium.com/@erika_dike/installing-the-protobuf-compiler-on-a-mac-a0d397af46b8
        
        ## Creating PROTOC Classes
        
        https://developers.google.com/protocol-buffers/docs/pythontutorial
        protoc -I=/Users/nicholasho/Documents/ClearGrid/commlib --python_out=/Users/nicholasho/Documents/ClearGrid/commlib /Users/nicholasho/Documents/ClearGrid/commlib/commlib/proto/location.proto
        
        # SETUP
        
        See `settings/settings.py` to set up the server settings
        The `client` and `server` libraries can also be invoked with arguments in the following order:
        
        ```
          server = UDP.SocketServerClass(
          SECRET_KEY, TSKDF_LENGTH, UDP_IP_ADDRESS, UDP_PORT_NO
          )
          client = UDP.Client(SECRET_KEY, TSKDF_LENGTH, UDP_IP_ADDRESS, UDP_PORT_NO)
        ```
        
        ## Constructing a Message
        
        Constructing a message requires `messageconstructor.py` and `payloadconstructor.py`
        
        This allows both the server, and the client to construct messages easily.
        
        1.  constructed_message = construct_message(collector, sequence, cipher, mac)
        2.  Create a payload_message `ping = PayloadConstructor.create_ping()`
        3.  augmented_payload = augment_payload(payload, 'ping', ping)
        4.  data = encrypt_and_sign_message(constructed_message, augmented_payload, cipher, mac)
        5.  self.sock.sendall(data)
        
        ## SERVER SETUP
        
        ```
        server = UDP.SocketServerClass(
                secret_key,
                TSKDF_LENGTH,
                bind_ip,
                bind_port,
                cipher,
            )
        ```
        
        Register any message handlers using the following:
        
        ```
        @server.register_dispatch("prepare_mission_message")
        ```
        
        # Running Commlib
        
        ## Client
        
        Client needs to initialize client by calling `client = commlib.UDP.Client()` with the following kwargs
        
        - secret_key,
        - tskdf_length,
        - udp_ip_address,
        - udp_port_no,
        - cipher,
        - mc_url,
        - collector_pk,
        - username,
        - password
        
        These following can be set by setting the Env Vars: `UDP_IP_ADDRESS`, `TSKDF_LENGTH`, `UDP_PORT_NO`, `MC_URL`, `MC_USERNAME`, `MC_PASSWORD`, `SECRET_KEY`, `COLLECTOR_PK`. These also have default variable settings.
        
        ### Initializing Information from MC
        
        The client attempts to acuqire information from MC through two calls:
        
        1. Getting the `mission_pks` for the collector (can be more than 1 mission active, although not advised)
        2. Initializing the `device_sync` for each `mission_pk`. This syncs the collector `device_ids` to what is reflected on the `server`
        
        ## Server
        
        Server needs to initialize client by calling `server = commlib.UDP.SocketServerClass()` with the following kwargs
        
        - secret_key,
        - tskdf_length,
        - udp_ip_address,
        - udp_port_no,
        - cipher,
        
        These following can be set by setting the Env Vars: `UDP_IP_ADDRESS`, `TSKDF_LENGTH`, `UDP_PORT_NO`. These also have default variable settings.
        
        ## Default Settings
        
        Default settings ensure that the `client` and `server` can communicate in a local setting.
        
        # Versioning
        
        This project follows Semantic Versioning except with how prereleases
        are encoded as python package indexes do not allow
        x.y.z-YYYY.MM.DD.HH.MM.SS. Instead we encode builds on master as beta
        releases and builds on every other branch as alpha releases and
        are encoded as x.y.zaYYYYMMDDHHMMSS
        
        # DevPi Setup
        
        ## DevPi Server Setup
        
        1. Install the Server: `pip install -U devpi-server` or `pip install -U devpi-web` (Recommend Devpi Web)
        2. Start and initialize the Server: `devpi-server --start --init`
        3. Start the server (after initialized): `devpi-server --start`
        4. Additional configuration for indexing/permanent installation....
        
        ## DevPi Client Setup
        
        1. Install the Client `pip install -U devpi-client`
        2. Start the Client `devpi use http://localhost:3141`
        3. Login `devpi login testuser --password=123`
        4. `devpi use testuser/dev`
        5. Install Packages `devpi install package`
        
        ## Setting Credentials after Initial Start
        
        1. `devpi login root --password ''`
        2. Set Root Credentials: `devpi user -m root password=123`
        3. `devpi user -c testuser password=123`
        4. `devpi login testuser --password=123`
        
        ## Uploading a Package
        
        1. Create an index that is not a mirror: `devpi index -c dev bases=root/pypi`
        2. Choose index `devpi use testuser/dev`
        3. NOTE: Only committed files will appear in a package!!!
        4. `devpi upload` in same directory as setup.py
        
        https://devpi.net/docs/devpi/devpi/stable/+doc/quickstart-releaseprocess.html#installing-devpi-client-and-server
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
