RATT

RATT can only be used in combination with TriplyDB. Contact info@triply.cc to receive your token to access the RATT package.

RATT is a TypeScript package that is developed by Triply. RATT makes it possible to develop and maintain production-grade linked data pipelines. It is used in combination with one of the TriplyDB subscriptions to create large-scale knowledge graphs.

Validating RDF output

RATT is able to automatically validate the RDF that is generated in the pipeline against a SHACL information model.

app.use(
  // Create all linked data statements.
  …
  // Now that all the data is created, validate it using a model.
  validateShacl(app.sources.model)
)

Validation report

Validation creates a report that is asserted in linked data. This report can be stored as a named graph in the created linked dataset.

The following example code stores the validation report in a dedicated named graph:

const prefix = {
  graph: 'https://triplydb.com/Triply/example/graph/',
}

const graph = {
  report: prefix.graph('report'),
}

app.use(
  // Create all linked data statements.
  …
  // Now that all the data is created, validate it using a model.
  validateShacl(
    app.sources.model,
    {report: {destination: app.sources.dataset,
              graph: graph.report}}),
)

Termination conditions

The validateShacl function can optionally be given the terminateOn option. This option determines when validation halts. It can take the following values:

'Never': Do not halt; run the validation for the full dataset.
'Violation': Halt validation when the first SHACL Violation is encountered.
'Warning': Halt validation when the first SHACL Violation or SHACL Warning is encountered.
'Info': Halt validation when the first SHACL Violation or SHACL Warning or SHACL Informational message is encountered.
undefined: Halt validation when the first SHACL message is encountered.

The following example code lets validation run for the full dataset, regardless of how many violations, warnings, and/or information messages are encountered:

app.use(
  // Create all linked data statements.
  …
  // Now that all the data is created, validate it using a model.
  validateShacl(app.sources.model, {terminateOn: 'Never'}
)

Log conditions

The validateShacl function can optionally be given the log option. This option determines when and which violations should be printed. The values are the same as in 'terminateOn' option. Note that log is about printing on your terminal and not about the violation report.

app.use(
  // Create all linked data statements.
  …
  // Now that all the data is created, validate it using a model.
  validateShacl(app.sources.model, {log: "Never"}
)

Upload prefixes

After loading the graphs, we can also upload other important elements in Linked data, such as the prefixes. This can be done by combining RATT functionality (app.after, app.prefix) with TriplyDbjs functionality (app.triplyDb.getOrganization, app.triplyDb.getUser() etc.).

You have to set the prefixes:

const prefix_def = Ratt.prefixer('http://example.com/def/')
const prefix_id = Ratt.prefixer('http://example.com/id/')
const prefix = {
  def: prefix_def,
  graph: prefix_id,
}

Then you have to include the prefixes in the RATT app:

export default async function(): Promise<Ratt> {
const app = new Ratt({
  prefixes: prefix,
  sources: {
  ..
  },

  destinations: {
    ..
  },
})
..
}

After finishing with the main body of the ETL and closing app.use(), you can use the below snippet to upload the prefixes under a specific organization, inside app.after.

app.after(
      async () => {
        const dataset0 =await (await app.triplyDb.getOrganization(organization)).getDataset(dataset)
        await dataset0.addPrefixes(mapValues(app.prefix, prefix => prefix('').value))
      })

You can upload the prefixes similarly under your account, using the relevant TriplyDbjs function. Also, note that mapValues is a function of lodash. For this reason, you will need to import it in the beginning of your script.

import { mapValues } from 'lodash'

Uploading graphs

In some cases, it is useful to upload graphs on TriplyDB that are already in a linked data format file. See copying source data.