Data Feed Tutorial
In Broadleaf's DataFeeds, you can create an entirely data-driven data feed for whatever platform you need, such as Google Products or Google LIA.
In this tutorial, we will create a basic Google Product which will make use of several types of DataFeedValues and create an XML file, which could be uploaded to your Google Merchant Account.
Prerequisites
You should be running in a Broadleaf instance that already has the DataFeed module configured.
Step 1, Create the Scheduled Job
First, we need to create a scheduled job. This relies on Broadleaf's ScheduledJobsAndEvents module. We'll add a job that will be trigged every 60 seconds and will generate a Google Product Feed XML file.
INSERT INTO BLC_SCHED_JOB (SCHED_JOB_ID, CRON_EXPRESSION, ENABLED, EXECUTED, NAME, TYPE, DATE_UPDATED) VALUES
(-124, '0/30 * * * * ?', 1, 0, 'Google Product Data Feed Tutorial', 'CUSTOM_DATA_FEED', current_date());
Step 2, Create a Data Feed
Next, we need to create a Data Feed that will be generated every time the scheduled job gets fired. Its NAME column must match the NAME column of the scheduled job. Please note, if you create several data feeds with the same name, only the first will actually be triggered by the scheduled job.
Also note, the value of EXPORT_FORMAT will determine the file extension type that is created.
INSERT INTO BLC_DATA_FEED (DATA_FEED_ID, CONTENT_TYPE_OVERRIDE, DISABLE_PUBLISH, DISABLED, ENTITY_TYPE, ERROR_LOG_PATH, FAIL_ON_ANY_INVALID_ITEM, NAME, OUTPUT_PATH, NAMESPACE_PREFIX, NAMESPACE_URL, EXPORT_FORMAT, JOB_ID) VALUES
(-40000, 'application/rss+xml', 0, 0, 'SKU', '/tmp/google-product-data-feed-tutorial-errors.log', 0, 'Google Product Data Feed Tutorial', '/tmp/google-product-data-feed-tutorial.xml', 'g', 'http://base.google.com/ns/1.0', 'RSS', -40000);
Step 3, Add Data Feed Fields
Now we need to specify what fields we want included for each item in our feed. The DTYPE field determines how the value for the corresponding field is calculated. We'll see how setting DTYPE comes into play in the next step of this tutorial. The REQUIRED field tells the item validator that an entire item should be excluded if the field does not have a valid value. The CONDITIONAL_RULE field can be set to an expression that makes the Data Feed Field required dependent on other values (for example, the size field is required if the item category type is Merchandise).
Note that a valid google data feed requires more fields than we are going to for this tutorial*
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40000, -40000, 'id', 'id', 0, 1, NULL, 'DataFeedFieldImpl');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40001, -40000, 'name', 'title', 0, 1, NULL, 'DataFeedFieldImpl');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40002, -40000, 'longDescription', 'description', 0, 1, NULL, 'DataFeedFieldImpl');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40003, -40000, 'product.manufacturer', 'brand', 0, 0, NULL, 'DataFeedFieldImpl');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40004, -40000, 'product.category.name', 'product_type', 0, 0, NULL, 'DataFeedFieldImpl');
Step 4, Create Dynamic Data Feed Values
If we want to pull the exact value for a specific attribute on an item. We will set the DTYPE to DataFeedFieldImpl.
Sometimes, we can't just pull an attribute straight off a item. We might need to edit it in some way.
There are currently four ways to alter values:
1. Static Values (StaticDataFeedFieldImpl) - We want to overwrite a value, regardless of what it was before, and always use a static value, which will be the same for every item. If your data has valid upc values, the gtin field should be a regular dataFeedField, but for this example we are using a static valid UPC value for all items.
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, STATIC_VALUE) VALUES (-40005, -40000, NULL, 'condition', 0, 0, 'get("product_type")=="Merchandise"', 'StaticDataFeedFieldImpl', 'new');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, STATIC_VALUE) VALUES (-40006, -40000, NULL, 'gtin', 0, 0, NULL, 'StaticDataFeedFieldImpl', '123456789012');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, STATIC_VALUE) VALUES (-40007, -40000, NULL, 'adult', 0, 0, NULL, 'StaticDataFeedFieldImpl', 'no');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, STATIC_VALUE) VALUES (-40008, -40000, NULL, 'gender', 1, 0, 'get("product_type")=="Merchandise"', 'StaticDataFeedFieldImpl', 'unisex');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, STATIC_VALUE) VALUES (-40009, -40000, NULL, 'age_group', 1, 0, 'get("product_type")=="Merchandise"', 'StaticDataFeedFieldImpl', 'adult');
2. Substituted Values (SubstitutedDataFeedFieldImpl) - We want to alter a value according to a regex pattern and a replacement pattern.
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, REPLACEMENT_PATTERN, REPLACEMENT_VALUE) VALUES (-40010, -40000, 'product.url', 'link', 0, 0, NULL, 'SubstitutedDataFeedFieldImpl', '.+', 'http://demo.broadleafcommerce.org$0');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, REPLACEMENT_PATTERN, REPLACEMENT_VALUE) VALUES (-40011, -40000, 'skuMediaXref(primary).media.url', 'image_link', 0, 0, NULL, 'SubstitutedDataFeedFieldImpl', '.+', 'http://demo.broadleafcommerce.org/admin$0');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, REPLACEMENT_PATTERN, REPLACEMENT_VALUE) VALUES (-40012, -40000, 'retailPrice', 'price', 0, 1, NULL, 'SubstitutedDataFeedFieldImpl', NULL, 'USD');
3. Category Values (CategoryFeedFieldImpl) - We want to map our internal category names to data feed-specific ones, such as Google's taxonomy. The Google Taxonomy can be found at:
https://support.google.com/merchants/answer/6324436?visit_id=1-636177744728718919-1977716767&rd=1
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE, CATEGORY_MAPPING_TYPE) VALUES (-40013, -40000, NULL, 'google_product_category', 1, 0, 'get("product_type")=="Merchandise"', 'CategoryDataFeedFieldImpl', 'GOOGLE');
INSERT INTO BLC_DATA_FEED_CATEGORY_MAPPING (DATA_FEED_CATEGORY_MAPPING_ID, CATEGORY_MAPPING_TYPE, TARGET_CATEGORY_NAME, CATEGORY_ID) VALUES (-40000, 'GOOGLE', '4614', 2002);
INSERT INTO BLC_DATA_FEED_CATEGORY_MAPPING (DATA_FEED_CATEGORY_MAPPING_ID, CATEGORY_MAPPING_TYPE, TARGET_CATEGORY_NAME, CATEGORY_ID) VALUES (-40001, 'GOOGLE', '212', 2003);
4. Availability Values (AvailabilityDataFeedFieldImpl) - We want to specifically go check the inventory of a item to see if it's in stock.
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40014, -40000, NULL , 'availability', 0, 0, NULL, 'AvailabilityDataFeedFieldImpl');
5. Product Option Values (ProducOptionDataFeedFieldImpl) -
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40015, -40000, 'color' , 'color', 1, 0, 'get("product_type")=="Merchandise"', 'ProductOptionDataFeedFieldImpl');
INSERT INTO BLC_DATA_FEED_FIELD (DATA_FEED_FIELD_ID, DATA_FEED_ID, SOURCE_PROPERTY_PATH, TARGET_PROPERTY_PATH, EXCLUDE_IF_EMPTY, REQUIRED, CONDITIONAL_RULE, DTYPE) VALUES (-40016, -40000, 'size', 'size', 1, 0, 'get("product_type")=="Merchandise"', 'ProductOptionDataFeedFieldImpl');
Run the Test
- Build and Start Admin
- Build and Start Site
Within 30 seconds, you should see that the /tmp/google-product-data-feed-tutorial.xml file has been created and contains all the fields of a standard Google Product data feed.