Core Concepts

The core concepts of Jayvee are pipelines, blocks, and value types.

Pipelines

A pipeline is a sequence of different computing steps, the blocks. The default output of a block becomes the default input of the next block, building a chain of computing steps. In the scope of a pipeline, you can connect these blocks via the pipe syntax:

pipeline CarsPipeline {
    // Assumption: blocks "GasReserveHttpExtractor", "GasReserveCSVInterpreter", "GasReserveTableInterpreter", and "GasReserveLoader" are defined

    GasReserveHttpExtractor
        -> GasReserveTextFileInterpreter
        -> GasReserveCSVInterpreter
        -> GasReserveTableInterpreter
        -> GasReserveLoader;
}

Blocks

A block is a processing step within a pipeline. It can have a default input and a default output. We differentiate the following types of blocks:

• Extractor blocks do not have a default input but only a default output. They model a data source.
• Transformator blocks have a default input and a default output. They model a transformation.
• Loader blocks do have a default input but nor a default output. They model a data sink.

The general structure of a pipeline consisting of different blocks is the following:

The common syntax of blocks is at its core a key-value map to provide configuration to the block. The availability of property keys and their respective value types is determined by the type of the block, called block type - indicated by the identifier after the keyword oftype:

block GasReserveHttpExtractor oftype HttpExtractor {
    // key: value
    url: "https://www.bundesnetzagentur.de/_tools/SVG/js2/_functions/csv_export.html?view=renderCSV&id=1089590";
}

In the example above, the url property of type text is defined by the corresponding HttpExtractor block type.

Blocks can be either defined as part of the language, called built-in or defined as composition of existing blocks by users in Jayvee, called composite block types. See the documentation for composite block types.

Value types

A value type is the definition of a data type of the processed data. Some blocks use value types to define logic (like filtering or assessing the data type in a data sink). We differentiate the following kinds of value types:

• Built-in value types come with the basic version of Jayvee. See built-in value types.
• Primitive value types can be defined by the user to model domain-specific data types and represent a single value. Constraints can be added to a primitive value types. See primitive value types.
• Compound value types: UPCOMING.

valuetype GasFillLevel oftype integer {
    constraints: [ GasFillLevelRange ];
}

constraint GasFillLevelRange on decimal:
    value >= 0 and value <= 100;

Transforms

Transforms are used to transform data from one value type to a different one. For more details, see transforms.

transform CelsiusToKelvin {
  from tempCelsius oftype decimal;
  to tempKelvin oftype decimal;

  tempKelvin: tempCelsius + 273.15;
}

Publishing / using model elements

If you want to use a model element in a different file other than the one you define it, you need to publish and use it.

1. Publish the element to make it usable in other files.

// Either publish right away when defining an element
publish constraint GasFillLevelRange on decimal:
    value >= 0 and value <= 100;

// Or define first and publish separately
constraint GasFillLevelRange on decimal:
    value >= 0 and value <= 100;

publish GasFillLevelRange;
publish GasFillLevelRange as PercentGasFillLevel; // Alias for renaming the published element

2. Use the element in another file

// Define from where you want to take elements

// Wildcard "*" makes all published elements of the file available
use * from './relative/path/to/file.jv';

// Named use only makes the named elements available
use { GasFillLevelRange } './relative/path/to/file.jv';
use { GasFillLevelRange as FillLevelRange } './relative/path/to/file.jv'; // Alias locally renames elements


// Then just use them as if they were defined on root level

valuetype GasFillLevel oftype integer {
    constraints: [ GasFillLevelRange ]; // GasFillLevelRange is defined in another file
}

Currently, only root-level elements can be published, so elements defined within a pipeline cannot be used in other files.