Only this pageAll pages
Powered by GitBook
1 of 11

1.0

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

2 Description

This section provides context for this Building Block.

The Information Mediator Building Blocking provides a gateway for exchange of data and services among GovStack Building Blocks through open-API rest-based interfaces to ensure interoperability and implementation of standards. The Information Mediator provides mechanisms for applications/Building Blocks to publish and consume services and event notifications among other GovStack Building Blocks.

Information Mediator services act as a channel through which Building Blocks and external applications can connect to services exposed by other Building Blocks such as registry services, identity services, and payment services. The Information Mediator Building Blocking provides a second service, as a broadcasting channel for notification of events among the connected applications in a Publisher-Subscriber (Pub/Sub) model. And also maintains a log of transactions (e.g., requests, events), as well as handling communication errors between Building Blocks and/or other applications via the Pub/Sub service. This component may employ other core components, such as registries, repositories, etc. By allowing different applications to exchange information, it can act as a mechanism to encourage or enforce best practices, data standards around Pub/Sub, and data-sharing policies in cross-facility workflows among business processes.

It is strongly recommended that a Building Block uses an Information Mediator for any communications across the internet. An Information Mediator is not required for communication between Building Blocks which are co-located.

The Information Mediator, through the “Service Access Layer” and the “Pub/Sub Layer” enables all use cases that make use of HTTP requests and the OpenAPI specification. For this exercise, we have tested against the use cases within the “Postpartum and Infant Care” and “Unconditional Social Cash Transfer” user journeys.

Information Mediator

Developed by Aleksander Reitsakas (Aktors, Estonia), Taylor Downs (OpenFn), Dr. P. S. Ramkumar (ITU), and Tõnis Pihlakas (AWS)

5 Cross-Cutting Requirements

This section will highlight important requirements or describe any additional cross-cutting requirements that apply to this Building Block.

5.1 GET and PUT APIs must be Idempotent (REQUIRED)

Application APIs will contain POST endpoints which are not idempotent. GET/PUT/DELETE APIs are idempotent relative to Information Mediator Building Block, but the idempotents of intermediated services depend on the service provider and cannot be specified at this level.

5.2 Databases should not Include Business Logic (RECOMMENDED)

Databases should not include business logic. We propose that this is a design recommendation which intends to make business logic all live in a clearly visible and accessible location. This may not always be followed, as the benefits of stored procedures (e.g., in reducing database transaction round-trips, etc.) may sometimes outweigh this general design recommendation.

5.3 Design for Asynchronous First (RECOMMENDED)

The Service Access Layer of the Information Mediator is synchronous first. The Pub/Sub Layer is asynchronous.

5.4 Use Standardized Data Formats for Interchange (REQUIRED)

Standard formats are used for communication with other Building Blocks. Inside Building Block non-standard protocols can be used.

5.5 Use Web Hooks for Callbacks (REQUIRED)

OpenAPI spec 3.0 is supported in the first version. Moving to OpenAPI 3.1 is planned for the future.

Performance Requirements

The Performance Requirements specify the basic parameters that an implementing government might use to establish performance requirements for scalability, throughput, and response times when reasonable/necessary. For example:

  • Minimum Throughput = 100 requests/sec.

  • Maximum Latency = 1 sec.

  • Concurrency = 1000 concurrent requests.

  • All solutions MUST be able to monitor and report, including but not limited to, resource consumption, throughput, latency, average latency, queue depth/backlogs, etc.

    • All of these indicators MUST be available through an administrative API.

    • The local monitoring agent should be configurable via web interface.

  • Retries and back-off strategies must be configurable.

The cross-cutting requirements described in this section are an extension of the cross-cutting requirements defined in the and the . This section will describe any additional cross-cutting requirements for this Building Block or differences with the Architecture Blueprint.

Ideally, all Building Blocks should be able to run a “monitoring agent” which handles reporting out logs, requests, Building Block-specific indicators, etc. to a monitoring service (e.g. )

Specific “” requirements are in the functional requirements section.

Architecture Blueprint document
Security Requirements
Datadog

9 Internal Workflows

This section provides a detailed view of how this Building Block will interact with other Building Blocks to support common use cases.

A workflow provides a detailed view of how the Information Mediator Building Block will interact with other building blocks to support common use cases. This section lists workflows that this building block must support. Other workflows may be implemented in addition to those listed.

9.1 Service Access

Example Sequence diagram for accessing service

The results of steps marked with '*' can be cached and reused between calls. In this diagram:

  1. Requestor makes a request to the local Information Mediator security server.

  2. Local Infirmation Mediator security server finds in the configuration where to send request.

  3. The Information Mediator signs & sends the request from application A to the security server for application B.

  4. The security server for application B receives the request, validates the signature, and then forwards it to the application/service/endpoint.

  5. The application/service/endpoint responds to request.

  6. The security server for application B signs the response and sends it back to the security server for application A.

  7. The security server for the application A validates the signature and forwards the response to application A.

  8. This is all synchronous. Application A’s request is open/unresponded until step 6.

9.2 Pub/Sub

9.2.1 Registering a Room

Room for Pub/Sub must be created before events can be described and processed. Room is created by its owner organization (Member), the organization responsible for the operation of events of a certain type.

Creation request should be confirmed by GovStack administrator.

9.2.2 Defining event type

Event type corresponds to a service description of a service accepting events for publishing and must be registered before usage. A single room might host multiple event types.

E.g. in India, "The Ministry Of Home Affairs" can be the "Owner" of a Room where events of type "emergency" are published by "citizens" or "entities" and entities such as "ambulance services", "Fire-fighting services", Hospitals, etc., can be Subscribers to this Room. In this scenario, an event of type "Fire emergency" can be published by any citizen enrolled in this Room. The Ministry can choose to have one "emergency" Room in each town and enroll Subscribers relevant to respective regions.

9.2.3 Subscribing to events of a certain type

  1. The Subscriber requests that events of certain types be delivered to them.

  2. The Subscriber specifies the desired delivery mode (push/pull). The Room and Subscriber conclude the delivery contract. Note that:

    • push delivery mode is when the Room sends events to the Subscriber’s API. In this case, Subscriber may request some QoS/SLA parameters of delivery.

    • pull delivery mode (OPTIONAL) means that the Room keeps a queue of events and the Subscriber can check that queue.

  3. At a minimum, this Building Block MUST provide for a push delivery mode. It may also provide a pull mode based on convenience.

Three endpoints to be declared per event type Pub/Sub instance:

  • an endpoint URL to be registered by every Subscriber on a per-event type basis.

  • (OPTIONAL) an endpoint URL on the Pub/Sub where all pull requests come from various Subscribers.

  • an endpoint in the Room to send messages for publication.

9.2.4 Event delivery

  1. The Publisher and the Room have a data access contract to establish trust for accessing information.

    • This is akin to the standard trust contract in the Information Mediator, the consumer must have rights to access a certain API.

    • Publishers have to sign a contract with the Room owner’s consent to gain access to specific Rooms and events. In the contract, Publishers declare what type of message will be published in a given Room.

  2. The Publisher generates an event:

    • The Publisher makes a POST call to the Room service of a particular event type.

    • The Room stores the event and replies with the event id. Event id is generated by Room or is taken from the original event dataset if provided by the publisher.

    • The Room can implement a first in, first out (FIFO) event distribution policy. If the Room provides FIFO guarantees, the Room may:

      • (a) assign a key to the event. If a key attribute is defined in the schema of the event, the value of that attribute will be used. Otherwise, the Room can assign a key at its own discretion.

      • (b) assign a sequence number to the event. The sequence number must be monotonically increasing for events that share the same key.

    • The Room sends an immediate acknowledgment to the Publisher.

  3. The Room distributes an event asynchronously as follows:

    • A reference to publisher and event id is added to the event dataset.

    • For each Subscriber:

      • (alt) If the mode is ‘push’, make a POST call to Subscriber GovStack service of the event type;

      • (alt) If the mode is ‘pull’, enqueue an event for request from the Subscriber;

        • There is a queue of events waiting to be processed per the Subscriber, such that the Subscriber might periodically check to see events waiting in their own queue, process those events, and clear the queue.

        • A pull mechanism is essential for resilience to network dropouts and traffic load balance at servers and for differentiating urgent/emergency events from normal events (this can be decided during implementation).

  4. (OPTIONAL - if mode is PULL) The Subscriber pulls an event:

    • The Subscriber makes a GET call to the Room service of the particular event type.

  5. (OPTIONAL) The Subscriber requests event details. Some event details may have more restricted regulations for handling and may be not included in event type. In this case, the Subscriber requests these details directly from the publisher by making a GET call to the referenced Publisher with event id as a parameter.

    • This call implies the existence of an associated contract between the Subscriber and the Publisher.

The Information Mediator Building Block creates a log of all messages published and distributed.

If an event payload is very large, then it is recommended to just publish the "event" and let Subscribers get full details directly from the Publishers as needed. It will simplify event payloads, and reduce Turnaround time (TAT), storage, and bandwidth significantly.

For example, “Clinic System A” wishes to broadcast data about a new birth so that it can be used to trigger asynchronous actions in several other systems. A Service is configured in Clinic System A that makes an HTTP POST request corresponding to the type of event and some valid JSON in the body to the Pub/Sub provider. The provider maintains a list of active “Subscribers” for that event type. Without manipulating the body, that message is effectively forwarded to each subscriber, so that REST services in the “Demographics Tracker”, “Insurance Registry”, and “Early Childhood Education” applications can consume that data and do things based on it.

It should be noted that the shape of the payload will be agreed upon beforehand so that the responsibility of being able to “ingest” a “newBirth” payload from Pub/Sub will fall on the REST service provider. Once an event is ingested, the responsibility to deliver the payload lies with the Information Mediator “room” based on the importance given to the event by a Subscriber. Thus, during registration of the Subscriber, parameters such as max queue depth, retries, failsafe mechanisms, and error handling have to be configured by each Subscriber as they subscribe to an event type.

1 Version History

The version history table describes the major changes to the specifications between published versions.

Version
Author
Comment

0.8

Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar

Initial revision

0.9

Aleksander Reitsakas

Technical Reviewers: Tony Shannon, Saša Kovačević, Riham Moawad, Riham Fahmi, Aare Laponin, Manish Srivastava, Palab Saha, Surendra Singh Sucharia, Arvind Gupta, Gayatri. P., Shivank Singh Chauhan, Gavin Lyons

External links updated

1.0 RC1

Aleksander Reitsakas, Taylor Downs, Dr. P. S. Ramkumar, Aleksa Krolls Reviewers: Neil Roy, Aare Lapõnin, Fergal Marrinan, Surendrasingh Sucharia, Amy Darling

Applied feedback from technical review, addressed formatting issues

1.0

Wesley Brown, Steve Conrad, Valeria Tafoya

Final edits to align content to specification template for GovStack 1.0 release

4 Key Digital Functionalities

Key Digital Functionalities describe the core (required) functions that this Building Block must be able to perform.

The Information Mediator Building Block should provide the following functionalities, organized across 6 different components or layers

4.1 Service Access

Service access is the facility through which different GovStack Building Blocks and applications may communicate securely with each other.

The service access layer should provide an administrative interface that can be used to define users and roles and what access each should have. This layer should also include a registration interface which allows new members to onboard to the Information Mediator.

The Service Access layer will also provide functionality to allow or restrict secure access to various resources, route requests to the correct providers, connect service requestors to providers, and enforce access privileges and security policies

4.2 Directory Services

The Directory service allows applications to discover resources or functionalities that are made available by the system. An application may query the Directory service to discover organizations, services, and APIs that are available to them

4.3 Pub/Sub Service

The Pub/Sub service is a facility through which applications may publish and subscribe to different events identified by unique message types, enabling more efficient and resilient communication and application design. It should provide broadcast/multicast capabilities to facilitate faster, more resilient application design.

4.4 Logging Services

The Information Mediator will manage a message log which will provide comprehensive audit trails for all services requests and access.

4.5 Monitoring Services

The Monitoring services will provide both operational and environmental reporting and dashboards, allowing users to discover the volume of requests to particular services or organizations as well as environmental reports on hardware/network usage.

4.6 Scaling/Throughput Services

The Information Mediator should support scaling services such as load balancers and allow for the addition of additional servers to manage increased system use.

Out-of-scope requirements

Note that the Information Mediator is not responsible for manipulating the payloads sent to and from various applications—in a sense, it is both the postal service and the roads/bridges/train tracks—but it does not read the contents of your mail.

The Information Mediator Building Blocking does not handle communication between human users and applications.

The Information Mediator Building Blocking does not handle ingress and egress access from external applications to and from GovStack’s internal components (This may be handled by a public API Gateway in the domain of the Security Building Block).

The current specification does not assume the possibility of different Information Mediator implementations working together. In one instance of the GovStack ecosystem, only one implementation of the Information Mediator Building Block can exist.

The following requirements have been identified as “out of scope” because they fall into the realm of the Workflow Building Block or the Security Building Block.

  • Supports the handling of transactions and associated communication errors and exceptions.

  • Translates data from one format to another, and interoperates with handshake protocols to enable interoperability between different ICT (Information and communication technology) Building Blocks duplications.

  • Map data structures and fields from the identification system to the registration system and vice versa.

  • Hold authentication and credentials for each system.

  • Allow the definition of steps for a particular transaction.

  • Provide an API for both systems to access – and execute all necessary steps for that transaction (including error handling, retries, and notifications).

  • Provide an API for external systems to access GovStack Building Blocks.

Future Scope

3 Terminology

Terminology used within this specification.

3.5 Service Access

3.6 Pub/Sub

Term
Description

Publisher

A Publisher produces events and sends them to rooms. Each event has an event type associated with it. Publishers can produce events of different types.

Room

A Room is a Pub/Sub entity that handles the distribution of events. Each Room has a set of connected event types (E.g., the “birth” Room might contain three event types: “new_birth”, “birth_complication”, and “infant_death”).

Subscriber

A Subscriber can process events of a certain event type. Subscribers are independent of each other and their business logic is different (as rule). Each subscriber processes events from their own perspective.

8 Service APIs

This section provides a reference for APIs that should be implemented by this Building Block.

The majority of functions provided by the Information Mediator Building Block are either defined in the “service access flow” or configured by the administrator via the web User Interface. There is, however, a “Directory Service” which can provide listings of clients, methods, and available API specifications for services on the Information Mediator. The directory is managed by admins of members. The directory service centralizes and offers knowledge of all enrolled members and their services along with the information necessary to bind a third-party application as a consumer of that service. These services are described here:

The services can be accessed via the following Service APIs:

8.1 Service Access

The full API definition of all available services can be produced from the set of all available OpenAPI descriptions. For that one need in all occurrences of:

“path”: { “<some-value>”: … }

prepend <some-value> with

{GovStackInstance}/{memberClass}/{member}/{application}/{service}/ extending this way the path with these details.

8.2 Directory Services

8.2.1 Member Discovery

At development time, to see which organizations are available on GovStack, an administrator of application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/listClients

The response is an array of organizations with descriptions. API MAY implement paging of output.

8.2.2 Service Discovery

At development time, an administrator at application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/INDIA/GOV/MEMBER/APPLICATION/{listMethods || allowedMethods}

The response is an array of services (either all services or services that the requester is authorized to access via “allowedMethods”). API MAY implement paging of output.

At development time, to learn about an available service, an administrator at application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/INDIA/GOV/MEMBER/APPLICATION/getOpenApi?serviceCode=SERVICE

The response is an OpenAPI specification, detailing the endpoints and requirements for that service/API of the requested Service of Application.

8.3 Pub/Sub Service

To broadcast a message to a Room, the service access API must be followed and the service requested must be the service implementing event type.

8.3.1 Subscriber API

8.3.2 PULL delivery mode API

8.3.3 Publisher API

8.4 Logging Services

At development time, to learn about an available service, an administrator at application A sends a GET request to the security server: url-of-local-information-mediator-security-server/r1/INDIA/GOV/MEMBER/APPLICATION/getOpenApi?serviceCode=SERVICE

The response is an OpenAPI specification, detailing the endpoints and requirements for that service/API of the requested Service of Application.

8.5 Monitoring Services

At the debugging time, to learn about system performance or retrieve an audit log, an administrator may send a request to the reporting API.

The response is <audit trail>, <metrics>, etc.

8.6 Management API

8.6.1 Configuration Management

8.6.2 Management of Access Rights

7 Data Structures

This section provides information on the core data structures/data models that are used by this Building Block.

7.1 Resource Model

The Mediator Building Block key element is Service. The Service is used by a consuming Building Block or an application and offered by a provider Building Block or Application. Both Provider and Consumer must be Members of the Mediator Building Block. Members of the Information Mediator Building Block can be an organization (governmental or not, business or not) or a person (citizen as a rule).

To become a Member of Mediator Building Block participants must fulfill declared requirements and apply for onboarding. In the process of onboarding a Member is registered with Mediator Building Block and gets credentials to connect to Mediator Building Block. Normally it is done the way that a Member provides a certificate of recognized Certification Authority (CA), and requests signed with this certificate are considered legitimate requests of the Member. A member entity can access the Pub/Sub configuration and register a room to publish its own event type as a publisher through its own admin.

Members can browse a directory of Services available in the Mediator Building Block. Each service is described in OpenAPI.

7.2 Data Elements

7.2.1 Member

Fields:

7.2.2 Application

Fields:

7.2.3 Service

Fields:

7.3 PubSub Layer

7.3.1 Resource Model

The Resource Model is an extension of the Access Layer model:

7.3.2 Data Elements

7.3.2.1 Event

An event is a message—a set of data sent to a topic. Each event has an id. The event corresponds to the message. Data elements of an event are described by event type OpenAPI description.

7.3.2.2 Event Type

An event type is schema definition for an event. Each event type is owned by a Room of a certain authority. (E.g. the Ministry of Health might own Room with the “new_birth” event type and define its schema.)

7.3.2.3 Publisher

A candidate application playing the role of IM-Publisher must be able to emit events to a specific Room.

7.3.2.4 Room

(N.B., this is often called a "topic" and we may shift to that in later versions.)

A candidate application playing the role of IM-Room must handle the distribution of events. Each Room has a set of connected event types (e.g., the “birth” room might contain three event types: “new_birth”, “birth_complication”, and “infant_death”). A room is located in the member’s local Information Mediator Building Block implementation and the member is responsible for all types of events in that particular room.

7.3.2.5 Subscriber

A candidate application playing the role of IM-Subscriber must be able to process events. It is done by defining a Service that will be called by Room to deliver an event. Subscribers are independent of each other and their business logic differs (as a rule). Each subscriber processes events from their own perspective.

7.3.2.6 Subscription

In future iterations of this specification, we may take into consideration more broad API-management standards which include multiple domains, such as those proposed by the United Kingdom's Government in their and the section on .

Term
Description

Building Block

Software modules that can be deployed and combined in a standardized manner. Each Building Block is capable of working independently, but they can be combined to do much more. Building Blocks are composable, interoperable software modules that can be used across a variety of use cases. They are standards-based, preferably open-source, and designed for scale. Each Building Block exposes a set of services in the form of REST APIs that can be consumed by other Building Blocks or applications.

Ingress access is access from external applications to GovStack Building Blocks and applications.

Egress access is access from within GovStack Building Blocks and applications to external applications.

Use Case

A use case is a piece of functionality described as a sequence of actions (steps) to achieve a specific goal in a specific context of usage. E.g., in one use case, the Information Mediator Building Block may be used to let a Building Block access a service provided by another, or in a different use case may be used to relay an event notification from one Building Block to several other Building Blocks via a Publisher-Subscriber (Pub/Sub) model.

Each use case may involve a collection of modules or Building Blocks. A relatively small set of these Building Blocks can be readily applied to a wide variety of use cases in low-resource settings.

API

An application programming interface () is a connection between computers or between computer programs. It is a type of software interface, that offers a service to other pieces of software. A document or standard that describes how to build such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation.

Term
Description

Member

A member is an organization that is authorized to communicate via the Information Mediator for a particular GovStack implementation.

Application

An application is a running instance containing one or more Building Block instances and zero or more use case implementations. An application uses an Information Mediator Building Block to communicate with other Building Blocks or applications. An application typically has a single responsibility.

Service

A service is a minimal piece of functionality provided by a Building Block or use case implementation. A service can be local (inside one application) or remote (between different applications). Remote services are consumed using REST protocol and they are described by specification. For example, the “registration” service might be accessed at a particular URL and allow a requester to “register a patient” by sending a POST request with proper patient data.

Security Server

A security server is the main piece of software that is responsible for implementing the “service access” layer of the Information Mediator. This software acts as a gateway and is responsible for mediating requests between various members, applications, and services. It might be not just a single piece, but also a clustered or serverless deployment.

The provides additional information on how 'adaptors' may be used to translate an existing API to the patterns described here. This section also provides guidance on how candidate products are tested and how GovStack validates a product's API against the API specifications defined here.

The tests for the Information Mediator Building Block can be found in .

and changes to the API definitions can be made by submitting a pull request on this repository. Additional APIs may be implemented by the Building Block, but the listed APIs define a minimal set of functionality that should be provided by any implementation of this Building Block.The is available.

The resource model shows the relationship between data objects that are used by the Information Mediator Building Block. The data elements provide detail for the resource model. All data element schemas can be viewed, commented on, and modified in the .

Name
Type
Description

Schema reference

Name
Type
Description

Schema reference

Name
Type
Description

Schema reference

Data Element
Default format
Description
Data Element
Default format
Description
Data Element
Default format
Description
Data Element
Default format
Description

Schema reference

Defining an API management strategy
Gov.UK’s API Management Strategy Document

class

enumerator

namespace for member_code, e.g. GOV or COM

code

string

unique id of member in registry defined by member_class

signing_key

string

name of key used to sign on behalf of Member

certificate

certificate

certificate of member issued by trusted CA

code

string

unique name of application in member scope

connection_type

string

protocol used for connection: http or https

certificate

certificate

TLS certificate

code

string

unique name of OpenAPI in scope of application

description_url

url

location of OpenAPI service description

service_url

url

service provisioning network address

ACL

object

description of access rights

id

string

application id of this publisher

name

string

OPTIONAL

id

string

application id of this room

name

string

OPTIONAL

id

string

application id of this subscriber

name

string

OPTIONAL

id

string

subscription id

room_id

string

room id

subscriber_id

string

subscriber id

event_type

string

A filter expression that allows a subscriber to subscribe to only certain message_types that are published to the room they're subscribed to.

mode

enum

delivery mode

details

object

details of delivery, like time to live, repetition policy, etc.

API
OpenAPI
GovStack non-functional requirements document
this GitHub repository
API metadata
Swagger variant
schemas section of the Information Mediator Building Block GitHub repository
member.json
application.json
service.json
broadcast.json

10 Other Resources

This section links to any external documents that may be relevant, such as standards documents or other descriptions of this Building Block that may be useful.

10.1 Key Decision Log

10.2 Future Considerations

. ​

.

A historical log of key decisions regarding this Building Block
A list of topics that may be relevant to future versions of this Building Block
Diagram source
Diagram source
Scaling/Throughput

6 Functional Requirements

This section lists the technical capabilities of this Building Block.

These functional requirements do not define specific APIs, they provide a list of information about functionality that must be implemented within the Building Block. Detailed design and feature lists of these blocks can be customized by developers to optimally match specific target implementation needs.

6.1 Service Access

6.1.1 Administrative Interface

  • There should be different types of administrative roles that provide different levels of access (RECOMMENDED)

  • There should be administrative roles on different levels of the ecosystem (RECOMMENDED):

    • global administrator role of the whole ecosystem,

    • local administrator role of Member,

    • an administrative role for configuring an Application.

  • There should be an ecosystem-level administrative interface that allows an Information Mediator administrator (with appropriate authentication) to register/deregister members and applications (RECOMMENDED)

  • There should be a security-server-level interface that allows an administrator for a single security server to manage members, applications, and services that live under a single security server (RECOMMENDED)

6.1.2 Registration (REQUIRED)

  • Registering a member/organization requires:

    • That the new member provides details for registration.

    • That the administrator verifies and accepts the request for registration.

  • Registering an application requires:

    • That the new member provides details for registration.

    • That the administrator verifies and accepts the request for registration.

  • Registering a service requires (in terms of API endpoints):

    • That the owner of the application provides:

      • OpenAPI 3.0 specification document (i.e., includes base paths like “/api/patients”, “/api/visits”, etc.).

      • Description.

      • List of enabled endpoints of OpenAPI specifications (the requirement is that we can enable/disable endpoints).

  • Managing a list of allowed consumers for services requires:

    • Application developers may access the directory service (described below) to select the services that they want to consume (OPTIONAL)

    • The request/approval/addition of allowed consumers process is a business-first process with manual steps (The hard part is negotiating data sharing agreements and signing contracts when that is completed the Information Mediator administrator can easily modify the allowed consumers' list manually)

    • An application must specify which member/application/service they want to access

    • The provider of that service must decide if the consumer is allowed to

    • Once approved, the requesting application will be added to the list of allowed applications for the requested service

6.1.3 Accessing Services

  • To make a request to another service via the Information Mediator, an application must(REQUIRED):

    • Using REST, make a valid HTTPS request to the local Information Mediator security server with headers that identifies itself at the application level.

    • The components of the request must be (with color code):

      • Security server URL;

      • API version;

      • Instance (e.g., Country);

      • Domain of member;

      • Member (e.g., Ministry of X);

      • Application;

      • Service (OpenAPI file);

      • Path

        • Endpoint.

        • Query parameters.

      Example Only Sample GET Request: url-of-local-information-mediator-security-server/r1/INDIA/GOV/ministry-of-agriculture-karnataka/market-linkages-app/inventory-service/v1/check-level/apples?fresh=true Response: { data: 7 } Sample POST Request: url-of-local-information-mediator-security-server/r1/INDIA/PVT/tata-buyers-corp-karnataka/small-farmer-buyers-app/inventory-service/v1/supply/apples With body: {“qtyAvailable”: 4} Response: { result: “Stock level report created.” } How to interpret the above request paths: SECURITY-SERVER-URL/r1/INSTANCE/DOMAIN/MEMBER/APPLICATION/SERVICE/PATH

    • Note that all applications are making requests to the security server, which runs over the private network segment, rather than making requests to other applications directly over the public internet. (This is one of the main points of the security server and Information Mediator architecture.)

6.2 Directory Services

  • At development time, to see which resources are available on GovStack, the administrator/developer of application A may send requests to the security server to see(OPTIONAL):

    • organizations with descriptions;

    • services (either all services, or services that the requester is authorized to access);

    • an OpenAPI specification, detailing the endpoints and requirements for the service/API.

  • A view layer allowing for easy exploration of ALL clients, applications, and services should be provided. (Note that, “under the hood”, this layer may make use of the APIs described above or be implemented via a separate API.)(RECOMMENDED)

6.3 Pub/Sub Service

6.3.1 Defining Rooms and Event Types

  • The Pub/Sub layer must provide a mechanism for registering Rooms (REQUIRED)

  • The Pub/Sub layer must provide a mechanism for registering event types (REQUIRED)

  • For each event type, an OpenAPI description of a service with JSON schema for the required payload shape must be defined to create an event of the type (REQUIRED)

  • An interface for registering and viewing event types must be provided (REQUIRED)

6.3.2 Publishing a Message

  • An application must be able to make a POST request with a valid JSON payload as the body and specify the message type to a Room provided by the Pub/Sub layer service (REQUIRED)

6.3.3 Delivering Messages

  • Published messages should be delivered to all subscribers (RECOMMENDED)

  • Published messages should be stored so that delivery may be done and retried asynchronously, e.g. if certain Subscribers are offline (RECOMMENDED)

  • If an active subscription exists but an error is received when forwarding a message to that service, the Pub/Sub layer should retry N times with a standard backoff. (An exponential backoff may be the default approach.) (RECOMMENDED)

    • The backoff and retry strategy should be configurable by an administrator with access to the Pub/Sub layer application

  • If a message cannot be delivered the Pub/Sub layer should drop that message (RECOMMENDED)

6.3.4 Managing Subscriptions

  • The Pub/Sub layer should provide an admin user interface to help create/manage subscriptions with the data below (RECOMMENDED)

  • The Pub/Sub layer should allow an administrator to view a list of active subscriptions (RECOMMENDED)

  • For registration via API, an application must be able to make a POST request to a service exposed by the Pub/Sub layer which defines which endpoints certain event types should be sent to (REQUIRED)

6.3.5 Event Logging and Audit Trail Generation

  • All events received and delivered must have a unique ID (REQUIRED)

  • All events received by the Pub/Sub layer must be logged or added to a log sync and those log entries must contain event metadata including the sender, timestamp, and event type, but may not include the event payload (REQUIRED)

  • All event delivery attempts must be logged or added to a log sync (REQUIRED)

  • For every event message received, the Information Mediator sends back an acknowledgment with the ID of the event to the respective Publisher (REQUIRED)

  • There must be a possibility to search and view the status of messages, for example, a message with type X to seven subscribers had been successfully delivered to all seven (REQUIRED)

6.4 Logging Services

  • The Information Mediator Building Block maintains a message log (REQUIRED)

    • The purpose of the message log is to provide means to prove the reception of a regular request or response message to a third party. Messages exchanged between Information Mediator Building Block are signed and encrypted. For every regular request and response, the security server produces a completely signed, and timestamped document. At a minimum, the log must store metadata that identifies a specific message, the status of transaction carried out on that message by Information Mediator Building Block, along with source ID and date time stamp.

  • The Information Mediator Building Block has full audit trail capabilities (REQUIRED)

    • The Information Mediator Building Block keeps an audit log. The audit log events are generated by the user interface when the administrator changes the system's state or configuration. The administrator's actions are logged regardless of whether the outcome was a success or a failure. The system must be capable of emitting statistical reports for a given organization, application, service, or consumer and status.

6.5 Monitoring Services

  • Operational monitoring (REQUIRED)

    • Operational monitoring provides details about the requested exchange, such as the ID-s of the client and the service, various attributes of the message read from the message header, request and response timestamps, sizes, etc., but not the actual payload of messages.

    • The operational monitoring daemon collects and shares operational monitoring data of the Information Mediator Building Block as part of request exchange, shares this data, and calculates and shares health statistics (the timestamps and the number of successful/unsuccessful requests, various metrics of the duration and message size of the requests, etc.).

    • The operational monitoring daemon makes operational and health data available to the owner of the security server, regular clients, and the central monitoring client via the security server. (For example, local health data may be made available for external monitoring systems.)

    • The owner of the security server and the central monitoring client are able to query the records of all clients. For a regular client, only the records associated with that client are available.

  • Environmental monitoring (REQUIRED)

    • Environmental monitoring provides details of the security servers such as operating system, memory, disk space, CPU load, traffic load, running processes, installed packages, etc. in a chosen date range.

    • Environmental monitoring provides a standard endpoint that can be accessed with a client (e.g. Java's console application if using Java Management Extensions).

    • It is possible to limit what allowed non-owners can request via environmental monitoring data requests. The security server owner will always get the full data set as requested.

6.6 Scaling/Throughput Services

  • The Information Mediator Building Block should support provider-side high availability setup if needed (RECOMMENDED)

  • Busy production systems may need a scalable performance in addition to high availability. The Information Mediator Building Block supports external load balancing mechanisms to address both of these problems simultaneously. A load balancer is added in front of a security server cluster to route the requests based on a selected algorithm (OPTIONAL)

  • The team in charge of deploying the security server application on their hardware must consider the network infrastructure including a load balancer, etc. The requirements for network infrastructure must be handled/defined in a (yet to exist) “Network Building Block”(REQUIRED)

The functional requirements section lists the technical capabilities that this Building Block should have. These requirements should be sufficient to deliver all functionality that is listed in the section. The Information Mediator is used only for interaction between Applications/Building Blocks for the transfer of data at an API level only. Details in the .

Open IAM (or another Identity and Access Management Solution) must be able to create/edit/delete admin users for the Information Mediator interface. (REQUIRED)

Subscribers should be able to configure their retry strategies, overriding the default exponential backoff on retriable errors ()

Key Digital Functionalities
Architecture and Nonfunctional Requirements document
Security Building Block specification 1.0.1 section 6-1
Google Pub/Sub example

Returns OpenAPI service description for a REST service

get
Path parameters
GovStackInstancestringRequired
memberClassstringRequired
memberCodestringRequired
applicationCodestringRequired
Query parameters
serviceCodestringOptional
Header parameters
X-GovStack-ClientstringOptional
Responses
200
OpenAPI description of the specified REST service
Responsestring
400
Error in request
500
Internal error
get
GET /r1/{GovStackInstance}/{memberClass}/{memberCode}/{applicationCode}/getOpenAPI HTTP/1.1
Host: securityserver
Accept: */*
text