Skip to main content
This page describes how Data Files are named, structured and formatted, how the files relate to each other, and how schema changes are managed. For what the service covers and how it is delivered, see the Data Files overview.

​
File naming

File names follow this pattern:
File names are lowercase throughout. For example:
The timestamp shows when the file was generated, not the day the data covers. Because files are generated the following morning, a file timestamped 7 September contains data for 6 September.

​
File structure

Data Files uses an event log structure. An event log is an append-only file in which each row represents one event related to the entity. Each daily file contains the events generated during that day. The files are not snapshots or current-state extracts. Because they are built from events, the same card, cardholder or token appears again in a later file whenever it is updated. Earlier files are never changed. Each event has a unique ID (card_event_id, cardholder_event_id, token_event_id or transaction_event_id), which you can use to identify each event within a file and across deliveries.

​
File format

Files are delivered in CSV format, using the following dialect. Under RFC 4180, a value containing a comma or a double quote is wrapped in double quotes, and any double quote inside the value is written twice. Standard CSV readers handle this automatically.

​
File schema

Each entity file follows an Enfuce standard schema. The schema defines the columns included in the file, their data types, descriptions and allowed values where relevant. It is based on the corresponding event payloads. The schema pages are the definitive list of columns. Only the fields listed there are included in the files:

​
Fields with multiple values

Fields that hold more than one value follow one of two conventions. Where the source schema defines a fixed maximum number of values, the field is flattened into separate columns. Where the number of values is variable, the field is delivered in a single column, so the column set stays the same in every file.

​
Schema changes

The schema evolves as the service develops. Changes are managed as follows:
  • New columns are added at the end of the file. Existing columns keep their order.
  • Breaking changes, such as renaming or removing a column or changing its data type or format, are announced in advance.
Read columns by their header name rather than by position, and ignore columns you do not recognise. Your ingestion then keeps working when new columns are added.

​
File relationships

The files are linked through stable identifiers. These identifiers show which records belong together, but they are not unique per row. Because the Card, Cardholder and Token files are event logs, the same card, cardholder or token appears on more than one row, once per event. Matching on the identifier alone therefore returns several rows rather than one. For example, a Transaction row matched on card_id returns every Card event row for that card. The column name differs between some files, so each side is named below.