Skip to main content
Version: Next

Core Concepts

The core concepts of Jayvee are pipelines, blocks, and value types.

Pipelines​

A pipeline is a sequence of different computing steps, the blocks. The default output of a block becomes the default input of the next block, building a chain of computing steps. In the scope of a pipeline, you can connect these blocks via the pipe syntax:

pipeline CarsPipeline {
    // Assumption: blocks "GasReserveHttpExtractor", "GasReserveCSVInterpreter", "GasReserveTableInterpreter", and "GasReserveLoader" are defined

    GasReserveHttpExtractor
        -> GasReserveTextFileInterpreter
        -> GasReserveCSVInterpreter
        -> GasReserveTableInterpreter
        -> GasReserveLoader;
}

Blocks​

A block is a processing step within a pipeline. It can have a default input and a default output. We differentiate the following types of blocks:

  • Extractor blocks do not have a default input but only a default output. They model a data source.
  • Transformator blocks have a default input and a default output. They model a transformation.
  • Loader blocks do have a default input but nor a default output. They model a data sink.

The general structure of a pipeline consisting of different blocks is the following:

The common syntax of blocks is at its core a key-value map to provide configuration to the block. The availability of property keys and their respective value types is determined by the type of the block, called block type - indicated by the identifier after the keyword oftype:

block GasReserveHttpExtractor oftype HttpExtractor {
    // key: value
    url: "https://www.bundesnetzagentur.de/_tools/SVG/js2/_functions/csv_export.html?view=renderCSV&id=1089590";
}

In the example above, the url property of type text is defined by the corresponding HttpExtractor block type.

Blocks can be either defined as part of the language, called built-in or defined as composition of existing blocks by users in Jayvee, called composite block types. See the documentation for composite block types.

Value types​

A value type is the definition of a data type of the processed data. Some blocks use value types to define logic (like filtering or assessing the data type in a data sink). We differentiate the following kinds of value types:

  • Built-in value types come with the basic version of Jayvee. See built-in value types.
  • Primitive value types can be defined by the user to model domain-specific data types and represent a single value. Constraints can be added to a primitive value types. See primitive value types.
  • Compound value types: UPCOMING.
valuetype GasFillLevel oftype integer {
    constraints: [ GasFillLevelRange ];
}

constraint GasFillLevelRange on decimal:
    value >= 0 and value <= 100;

Transforms​

Transforms are used to transform data from one value type to a different one. For more details, see transforms.

transform CelsiusToKelvin {
  from tempCelsius oftype decimal;
  to tempKelvin oftype decimal;

  tempKelvin: tempCelsius + 273.15;
}