Product Catalog in MAD format

The product feed is the necessary input for the vue.ai engines. For the first time processing, we require a one-time bulk data load. After this, product updates can be sent in real-time or at a regular cadence for us to update our systems.

Bulk data load:

The first time the catalog is loaded into the system, all the products are given as one or more files split into multiple batches for manageable file sizes. The files could also be compressed to save space and bandwidth.

Updates or Delta data load:

New products or updates to existing products can be sent as messages to an AWS SQS endpoint. Each product will be sent in a separate message. Alternatively, the updates can be sent at a regular cadence in any of the formats mentioned below.

  • Any product with a new ID will be considered a new product to the catalog
  • Any product with an existing ID, will be considered an update to the existing product in the catalog. All the existing information will be overwritten
  • If the product has to be removed from the catalog, the “available” field should be set to false and sent with the correct ID

We support product feeds in the following formats:

  • SQS
  • JSON
  • CSV
  • XML

Each product must have the following fields:

Key Type Description
id String Unique ID for the product
parent_id String Parent ID for the product (if multiple variants are available) [optional]
title String Name or Title of the product
brand String Brand name [optional]
image_link String URL for the primary image
small_image_link String URL for the thumbnail image [optional]
link String URL for the product display page
price Float Source site for the product if f the catalog is from different sites. [optional]
description String Short description of the product [optional]
gender String Intended gender of the product if available [optional]
category String The top level category for the product
sub_category String The second level category for the product [optional]
sub_sub_category String The third level category for the product [optional]
available Boolean Shows if the product is in stock/out of stock [optional]
ontology String Specifies the ontology hierarchy formed with gendercategorysub_categorysub_sub_category separated by “>” symbol. It is converted into lowercase and all special characters replaced with an underscore. By passing in the “ontology” field, the default setting is overridden. [optional] Eg: gender > cat > sub_cat> sub_sub_cat
date Datetime When the message is enqueued in SQS in “YYYY-MM-DD HH:MM:SS” format.

Bulk data load:

The first time the catalog is loaded into the system, all the products in the catalog is given as one or more files split into multiple batches for manageable file sizes. The files could also be compressed to save space and bandwidth.

JSON File:

A single JSON file can have multiple products. Each line in the file will represent a single product. Each line is a valid JSON object and will be deserialized line-by-line.

Example JSON object for 1 product (with indentation for readability):

{"id":"123",

"title":"Product Title or Name",

"brand":"Brand Name",

"image_link":"http://www.example.com/static/image_123.jpg",

"small_image_link":"http://www.example.com/static/small_image_123.jpg",

"link":"http://www.example.com/product/123/",

"source":"site/source",

"price":28.49,

"description":"A short description of the product. ",

"gender":"Women",

"category":"Dresses",

"sub_category":"Evening Dresses",

"sub_sub_category":"Little Black Dress",

"available":true,

"ontology":"women>dresses>evening_dresses>little_black_dress",

"date":"2016-06-15 15:45:47"}

CSV File:

The catalog can be given in the form of a CSV file also. The field names for the CSV files are same as mentioned above. In each CSV file, the first line will have the field names and each product will be represented by a single row in the CSV file.

Updates or Delta data load:

New products or updates to existing products can be sent as messages to an AWS SQS endpoint. Each product will be sent in a separate message.

  • Any product with a new unique ID will be considered as a new product to the catalog.
  • Any product with an existing ID, will be considered as an update to the existing product in the catalog. All fields will be replaced with the new product information.
  • If the product has to be removed from the catalog, mark the “available” field as false and send the product with the correct ID.
  • 70
  • 29-Apr-2019
  • 144 Views