TIBCO Scribe® Platform API Connector Sample — Dependencies

The TIBCO Scribe® Platform API Connector follows the Acyclic Dependencies Principle, which states that "the dependency graph of packages or components should have no cycles", or no circular dependencies.

  • The diagram above displays the acyclic dependency graph of the namespaces from the production assembly.
  • There is no circular dependency between any types in the whole solution.

The Connector uses the following third-party NuGet packages:

The Connector consists of four parts:

  • Metadata — Contains reusable types for providing metadata, traversing expression trees, and validating comparison expressions against metadata.
  • HTTP — Contains reusable types for working with any HTTP Web API.
  • Domain — Contains domain Plain Old CLR Objects (POCO), which are the metadata source, and they are annotated with metadata attributes. Also contains Data Transfer Objects for TIBCO Scribe® Platform API Connector.
  • Application — Glue code from the root namespace that connects reusable code pieces

    Key Application Concepts —

    • It is the Application that configures and uses library types from other namespaces.
    • Knows the details of how to work with the TIBCO Scribe® Platform API Connector. Types from other namespaces know nothing about the Connector and API specifics.
    • Contains “business” logic.
    • All the workarounds are contained here.
    • ScribeApiConnector class is an Entry Point and a Composition Root.

Metadata

Attribute-Based Metadata Provider

Metadata namespace contains following attributes:

  • SupportedActionAttribute : Attribute, IActionDefinition
    • Valid on: Classes, Structs, Interfaces
    • Supports multiple uses on the same program element
    • Does not support inheritance
  • ObjectDefinitionAttribute : Attribute, IObjectDefinition
    • Valid on: Classes, Structs, Interfaces
    • Does not support multiple uses on the same program element
    • Supports inheritance
  • PropertyDefinitionAttribute : Attribute, IPropertyDefinition
    • Valid on: Fields, Properties
    • Does not support multiple uses on the same program element
    • Supports inheritance

Key Concept: Attributes implement interfaces from Scribe.Core.ConnectorApi.Metadata, so the usage on program elements can be returned as metadata.

AttributeBasedMetadataProvider is an attribute processor that scans types from the given EntityAssembly annotated with metadata attributes according to the TypeFilter.

Here is an algorithm that approximates how it works:

  1. RetrieveActionDefinitions:
    1. Select from given EntityAssembly all types which are:
      1. annotated with ObjectDefinitionAttribute
      2. AND annotated with at least one SupportedActionAttribute
      3. AND passed TypeFilter
    2. Return all unique SupportedActionAttributes (by FullName and Description) 2.
  2. RetrieveObjectDefinitions:
    1. Select from given EntityAssembly all types which are:
      1. annotated with ObjectDefinitionAttribute
      2. AND passed TypeFilter
    2. Return all ObjectDefinitionAttribute with:
      1. SupportedActionFullNames from SupportedActionAttributes
      2. PropertDefinitions from PropertyDefinitionAttribute usages on properties or fields

If you do not specify a name in the attribute, then:

  • For a supported action the name an annotated type is used
  • For an object definition the name of an annotated type is used
  • For a property definition the name of a field/property is used

The current implementation does not support relationship and method definitions.

Expression Traversing

ExpressionVisitor traverses expressions that come as query constraints or operations. As its name indicates it is a modification of the Visitor pattern with result accumulation. In most implementations Visitor returns nothing, i.e. void. As a result it returns a list of candidates for the search. Each candidate is just a string-based dictionary where the key is a property definition name, and the value is the right value from the comparison expression.

ExpressionVisitor supports:

  • AND and OR logical operator
    • For OR it provides feature toggle which can be removed if support for this operation is needed
  • EQUAL comparison operator

ExpressionVisitor works like a sequence of demultiplexers and implements multiple dispatch. The second one is not really a demultiplexer, because it produces signals on both of the outputs.

Validation

CanidateValidator —

  • Validates the input candidate to query against metadata from injected IMetadataProvider using built-in TypeConverters in the .NET Framework TypeDescriptor.
  • Candidates can come as a result of the visiting expression by ExpressionVisitor.
  • Returns a list of validation errors grouped by candidate key, such as property name.

Domain

Domain types are just POCO ’s which are:

  • The source of metadata
    • types annotated with ObjectDefinitionAttribute are object definitions
  • Data Transfer Objects for the TIBCO Scribe® API
    • ScribeApiClient uses them to serialize requests and deserialize responses from and to the TIBCO Scribe® API

HTTP

The Connector uses HttpClient from System.Net.Http as the client for the TIBCO Scribe® API, for the following reasons:

  • It is part of .NET Framework and we do not need an extra dependency.
  • It is testable. HttpClient can consume HttpMessageHandler in the constructor, for which a developer can provide Test Double (stub, mock, fake, etc.) in unit tests.
  • It has an extendable pipeline by passing a custom HttpMessageHandler.
  • It supports composition by using DelegatingHandler.

Serialization

SingleMediaTypeHttpClient wraps an instance of HttpClient and uses the same MediaTypeFormatter for serializing requests and deserializing responses from or to any HTTP Web API. This type provides safe communication with HTTP.

TIBCO Scribe® Platform API Client

ScribeApiClient is a static class that contains a set of asyncextension methods for SingleMediaTypeHttpClient, which are mapped one-to-one to TIBCO Scribe® Platform API methods.

ScribeApiClient.Create is a Static Factory Method that returns the HTTP client to work with the TIBCO Scribe® Platform API, configured with the following:

  • Default headers for each request
  • Basic Authentication
  • JsonMediaTypeFormatter
    • Single configuration point for Json serialization
    • All other code should avoid direct usage of Newtonsoft.Json
  • One second delay for each response

API Limits

To avoid exceeding API Limits, each instance of the Connector sends a request to the TIBCO Scribe® Platform API no more than once per second. See DelayedHttpMessageHandler in the source code for the Connector.

Limitations

  • The Connector does not contain any retry logic.
  • It is possible to have a race condition:
    • Multiple TIBCO Scribe® Platform API Connections with the same credentials are counted against the same API limits.
    • API limits are an implicit shared mutable state between these Connections.

Pagination

For each API resource that supports pagination, ScribeApiClient implements lazy evaluation of each response page via Iterator methods. For more information see, yield. By default, it uses 100 as a page size.

Unfortunately, yield and await keywords cannot be used in the same method. To generate an understandable exception message, ScribeApiClient calls the following: 

Task.GetAwaiter().GetResult() instead of Task.Result :
// DO: will throw non-AggregateException on error
var page = retrievePage(offset).GetAwaiter().GetResult();
// AVOID: will throw AggregateException on error with
// "One or more errors occurred" message
var page = retrievePage(offset).Result;

Response Analyzer

SingleMediaTypeHttpClient provides an extension point for additional response analysis. The ScribeApiResponseErrorAnalyzer.ThrowOnFailureStatusCode extension method analyzes HttpResponseMessage from the TIBCO Scribe® Platform API. It knows the details of possible API response message formats and how to overcome some associated issues.

For failed responses, such as for non-2xx HTTP statuses, it generates an HttpRequestException with the following message format:

$"{ ( int ) response.StatusCode } ({ response.ReasonPhrase }) { errorDetails }"

Where errorDetails include:

  • plain/text — Returns the whole response body
  • application /json —
    • Tries to deserialize JSON to model errors and convert them to single error message
    • If it fails nothing is returned because the API has an issue.

      When accessing a resource that does not exist, TIBCO Scribe® Platform API can return a 404 response with a plain text error but "application/json" content type.

  • other — Returns nothing, because the API can return the whole HTML page in response.

For successful responses this method quickly returns nothing.

Cross-Cutting Concerns

Logging And Error Handling

Logger calls are not randomly placed in the whole code base. Logger calls are used only in the following places:

  • Direct calls of ScribeLogger in ScribeApiConnector and ScribeApiOperationExecutor in catch blocks
  • In LoggingMetadataProvider

All other code generates exceptions with readable messages and exceptions are handled and re-generated in Composition Root.

LoggingMetadataProvider is a Decorator that adds logging ability to any implementation of IMetadataProvider.

Caching

Since the Connector’s metadata does not change frequently, it is natural to cache. CachingMetadataProvider is a Decorator that adds caching ability to any implementation of IMetadataProvider . To check that caching really improves performance, run CachingMetadataProviderBenchmark .

To have attribute-based metadata provider with logging and caching do the following:

this .metadataProvider =
new LoggingMetadataProvider (
new CachingMetadataProvider (
new AttributeBasedMetadataProvider (
Assembly .GetExecutingAssembly(),
t => t.Namespace == typeof ( Organization ).Namespace)),
ConnectorName);

Instead of putting all of the logic for logging, caching, and metadata creation in single class, it is better to split it into multiple reusable types. Decorators reveal the true power of Composition or Aggregation.

 

See

TIBCO Scribe® Platform API Connector Sample

TIBCO Scribe® Platform API Connector Sample — Solution Structure

TIBCO Scribe® Platform API Connector Sample — Build