Stuck? Ask us for Help!

Fanouts are powerful, but can look daunting. Don't hesitate to ask your customer success agent for help with setting up a fanout script that fits your needs.

TODO additional text types

Fanout

The fanout script is the most powerful part of your pipeline. It allows you to dynamically define and control how each data object generates text, and how many generations happen per object. You can define what languages to generate, what blueprints to use, and how to deliver the resulting text generations.

Each pipeline has one fanout script, which is executed for each object in the data storage if the object is schedules for generation (wether you enabled automatic generation or a user has pressed "Generate" manually).

Just like the preprocessor, the fanout script is written in JavaScript. The simplest fanout script that just generates a text in every language and with every blueprint in your pipeline looks like this:

export default function (object, { languages, blueprints, dataPools }) {
	const jobs = []
	for (const language of languages) {
		for (const blueprint of blueprints) {
			jobs.push({
				blueprint: blueprint.id,
				language
			})
		}
	}
	return jobs
}

The pipeline takes the render jobs that you have returned from your fanout script and starts a text generation for each one, with the specified blueprint and language.

Data Pool Queries & Blueprint Enrichment

Two features that empower you to combine multiple data pools and use them in your text generation are data pool queries and blueprint enrichments. Via the dataPools object in the context argument of the fanout script, you can access data from any data pool and use it to enrich your blueprint jobs by including additional information.

export default function (object, { languages, blueprints, dataPools }) {
	// get supplementary brand data from the brand data pool matching the brand id of the object
	const brand = dataPools.brands.findOne({ uid: object.brandId })
	…
	// add brand data to the render job
	jobs.push({
		blueprint: blueprint.id,
		language,
		enrichments: {
			brand
		}
	})
	…
}

Currently we offer these functions to query data pools:

• findOne(query): searches for a single object in the data pool that matches the given query
• findMany(query): searches for multiple objects in the data pool that match the given query (up to 25)

Queries are a subset of the MongoDB query language.

You can then add this queried data (or any other variables you create in script) to the render job by adding it to the enrichments object. This will make the data available in the blueprint via the enrichments node.

More to Come

We will also add an aggregate query to data pools soon, which allow to gather statistics and other aggregated data from the whole data pool, like average prices or most common categories.

Webhook Delivery

After your texts have successfully been generated, you can deliver them to your systems via webhook. You can specify a webhook URL in the render job definition, and the pipeline will send a POST request to that URL with the generated text.

export default function (object, { languages, blueprints, dataPools }) {

	jobs.push({
		blueprint: blueprint.id,
		language,
		delivery: {
			url: `https://your-webhook-url.com?language=${language}`,
			`hmacSecret`
		}
	})
	…
}

The webhook config for each job can be different, so you can choose a different URL for different text types for example. You can also specify query parameters in the url to pass additional information to your webhook that is otherwise only available in the fanout script.

You can also sign the webhook payload with a secret to ensure that the payload is coming from AX. The hmacSecret is used to create a sha1 HMAC signature of the payload, which is then sent in the X-Webhook-Signature header of the request by default. You can verify the signature on your end to ensure the payload is authentic.

TODO how to verify the signature

Fanout Interface

The fanout script needs to export a default (optionally async) function that takes two arguments:

• object: The object that content is being generated for. This can come from any of your data pools, but in practice will most likely be from your primary data pool. This object is read-only, any changes you make to it will not be forwared to the blueprint.
• context: An object with the following properties (you can further destructure this):
  • languages: An array of strings of language codes that you have enabled in your pipeline.
  • blueprints: An iterable dict of blueprint objects keyed by label that you have created in your pipeline. A blueprint object looks like this:
    • id: The blueprint id
    • label: The blueprint label
  • dataPools: A dict of data pool objects keyed by id that you have created in your pipeline. A data pool object looks like this:
    • id: The data pool id
    • label: The data pool label
    • query functions

The fanout function needs to return a list of render job definitions. A render job definition is an object with the following properties:

• blueprint required: The id of the blueprint or a blueprint object itself to use for this render job
• language required: The language code to generate the text in
• enrichments optional: An object with arbitrary additional data to enrich the blueprint with
• delivery optional: An object with delivery configuration for this render job
  • url required: The URL to send the generated text to
  • hmacSecret optional: A secret to sign the webhook payload with
  • hmacHeader optional: Additional headers to send with the webhook request, default: 'X-Webhook-Signature'

Formal Definition (don't worry if you don't understand this)

export default async function (object: Object, context: FanoutContext): RenderJob[]

type FanoutContext = {
	languages: string[],
	blueprints: { [label: string]: Blueprint } | Iterable<Blueprint>
	dataPools: { [id: string]: DataPool }
}

type BlueprintId = string

type Blueprint = {
	id: BlueprintId,
	label: string
}

type DataPool = {
	id: string,
	label: string,
	findOne(query: object): Object | null,
	findMany(query: object): Object[]
}

type RenderJob = {
	blueprint: BlueprintId | Blueprint,
	language: string,
	enrichments?: object,
	delivery?: {
		url: string,
		hmacSecret?: string,
		hmacHeader?: string
	}
}