How a Product Sync Automation Project Transformed Customer Onboarding
When people think about impactful engineering work, they often imagine distributed systems, high-scale infrastructure, or complex algorithms.
One of the most impactful projects I worked on wasn't any of those.
It was solving a seemingly simple problem:
Keeping product data in sync across multiple retail systems.
Years later, our CEO still remembers how much smoother customer onboarding became after this project.
The Context: What is Commerce Connect?
At Casa Retail AI, we have an internal platform called Commerce Connect (CC).
Commerce Connect acts as the central Product Information Management (PIM) system and serves as the source of truth for product information.
Under the hood, it is built on top of a customized version of the open-source e-commerce platform Spree Commerce, extended with multi-vendor and multi-tenant capabilities.
Its primary responsibility is simple:
Collect product information from multiple retail ecosystems and distribute it to every Casa product that needs it.
Once product data enters Commerce Connect, it is synchronized to multiple downstream systems.
Why Product Data Matters
Many applications inside Casa depend on product information.
Product Consumers
Once product data enters Commerce Connect, it is distributed to multiple systems across the Casa ecosystem.
Customer-Facing Applications
Several products rely on product information to provide context and improve customer experience:
- Lead management applications use product information during customer interactions.
- Ticket management systems link customer issues to specific products.
- Digital receipts display product names, images, and related details.
Analytics & Reporting
Product data powers business dashboards and reports, helping retailers answer questions such as:
- Which categories perform best?
- Which products attract the most attention?
- Which products generate the most complaints?
It is also used for filtering and segmentation across various dashboards.
External Integrations
Some tenants require product synchronization with external platforms, including:
- B2B applications
- E-commerce experiences
- Digital catalog/flipbook solutions
- Custom third-party integrations
Because multiple systems depend on the same product catalog, maintaining a single source of truth is critical.
This ensures every system across the ecosystem works with consistent and up-to-date product information.
The Original Process
Whenever a new tenant was onboarded, the first challenge was importing their entire product catalog.
Most retailers maintained product data in POS systems such as:
- SAP
- Wondersoft
- APX
- Other proprietary retail platforms
The onboarding process looked like this:
- Customer Success (CS) teams collected product exports from the tenant.
- The tenant provided CSV files containing product information.
- CS teams manually mapped CSV columns to Commerce Connect fields.
- The CSV was uploaded into Commerce Connect.
- Each upload created a batch task.
- Background workers processed the batch and created products inside Commerce Connect.
- After products were created, additional synchronization jobs were triggered for:
- PostgreSQL
- ClickHouse
- B2B applications
- Other enabled integrations
Visually, the flow looked something like:
POS Export
↓
CSV File
↓
Manual Mapping
↓
Commerce Connect
↓
Batch Processing
↓
PG / ClickHouse / B2B Sync
At first glance, this seemed reasonable.
But as the business scaled, the cracks started to show.
The Problems We Started Seeing
1. Product Data Became a Bottleneck
A tenant could not fully use the platform until their product catalog was available.
This meant onboarding timelines were heavily dependent on receiving CSV files from the customer.
Sometimes the customer took days to provide them.
Sometimes the files were incomplete.
Sometimes they contained incorrect mappings.
Engineering wasn't the bottleneck.
Product data was.
2. Keeping Products Updated Was Painful
Product catalogs constantly change.
New products get added.
Existing products get updated.
Products become inactive.
All of these changes happen in the retailer's POS system.
Whenever a tenant changed something in their POS, Customer Success teams had to:
- Request updated exports.
- Receive new CSV files.
- Upload them again.
- Reprocess the entire catalog.
Keeping thousands of products synchronized through spreadsheets quickly became operationally expensive.
3. Every Tenant Required Manual Configuration
Different POS providers generated different CSV formats.
The Customer Success team had to repeatedly configure mappings for every upload.
The process was repetitive, error-prone, and difficult to scale.
The Key Observation
The product data already existed inside the POS systems.
Why were we asking humans to manually export and upload it?
Why not fetch it directly from the source?
The Solution: Pull Products Directly From POS Providers
SAP was the first provider that offered APIs for product retrieval.
Instead of waiting for CSV exports, we integrated directly with their APIs and automatically imported products into Commerce Connect.
The result was immediate.
No spreadsheets.
No manual exports.
No repeated uploads.
Just product synchronization directly from the source system.
After the success of the SAP integration, we expanded the same approach to other POS providers.
Designing for Scale
One requirement was clear:
We didn't want to rewrite the entire synchronization pipeline every time a new POS provider was added.
So we introduced a provider abstraction using a Factory Pattern.
Factory.build("sap_iplanet")
The factory dynamically resolves and instantiates the appropriate provider implementation.
Each provider only needs to answer a few questions:
- How to authenticate?
- How to fetch products?
- How to transform data?
Everything else remains shared.
POS Provider
↓
Provider Adapter
↓
Commerce Connect
↓
Existing Sync Pipeline
This gave us a huge advantage.
Adding a new POS integration became as simple as creating a new provider class.
The provider automatically inherited:
- Batch processing
- Error handling
- Task creation
- Product synchronization
- PostgreSQL sync
- ClickHouse sync
- B2B integrations
The integration surface became dramatically smaller.
Eliminating Repetitive Mapping Work
The old CSV process required Customer Success teams to repeatedly map columns.
For example:
CSV Column → Casa Field
item_code → SKU
item_name → Product Name
dept_name → Category
Every upload carried the risk of mapping mistakes.
With APIs, we could see the complete product schema exposed by the provider.
The mapping became a one-time configuration.
Provider Field → Casa Field
item_code → SKU
item_name → Product Name
dept_name → Category
After configuration, all future synchronizations happened automatically.
This significantly reduced operational effort.
More Control Over Synchronization
The CSV approach only supported full catalog uploads.
Whether one product changed or ten thousand products changed, the entire file was processed again.
With API-based synchronization, we gained much more control.
We could:
- Pull products on a schedule.
- Fetch only relevant updates.
- Track synchronization status.
- Improve error visibility.
- Retry failed imports independently.
The synchronization process became far more predictable and manageable.
Solving the "I Can't Wait Until Tomorrow" Problem
While nightly synchronization worked for most tenants, some businesses needed updates immediately.
Waiting until the next scheduled pull was not acceptable.
To solve this, we introduced a product upsert API.
Now tenants had two options:
Pull Model
Commerce Connect fetches products from the provider daily.
Provider → Commerce Connect
Push Model
External systems push product updates directly into Commerce Connect whenever changes occur.
Provider → Product API → Commerce Connect
This hybrid model gave tenants the flexibility they needed while preserving a single source of truth.
Operational Reliability and Monitoring
Automation is only valuable if it is reliable.
Once product synchronization became dependent on external POS APIs, we had to account for situations outside our control:
- API credentials changed
- Access tokens expired
- Provider endpoints became unavailable
- Network failures occurred
- Providers returned unexpected response formats
- Product payload validation failures happened
To ensure synchronization issues never went unnoticed, we built proactive monitoring and alerting into the pipeline.
Whenever a synchronization task encountered unexpected failures, automated Discord alerts were triggered with relevant diagnostic information, including:
- Tenant name
- Provider name
- Failure reason
- API response details
- Task identifiers
This allowed the Customer Success team to immediately investigate whether the issue was caused by invalid credentials, provider-side outages, configuration changes, or data quality problems.
Instead of discovering synchronization failures days later through customer complaints, the team could identify and resolve issues proactively.
The result was a much more reliable synchronization platform and significantly reduced operational risk as the number of integrated tenants continued to grow.
The Business Impact
From a technical perspective, this wasn't the most complex system I've built.
There were no distributed consensus algorithms.
No cutting-edge infrastructure.
No massive scale challenges.
But the impact was real.
Customer onboarding became significantly smoother.
Customer Success teams no longer spent hours chasing spreadsheets.
Product synchronization became reliable and automated.
New POS integrations became faster to implement.
Operational effort reduced dramatically.
Most importantly, the business could scale without product data becoming a bottleneck.
What I Learned
One lesson has stayed with me throughout my career:
The most valuable engineering work is not always the most technically sophisticated work.
Sometimes the highest-leverage solution is simply removing friction from a process people repeat every day.
At the time, this felt like a small automation project.
Today, many of the people using the platform don't even realize the problem it solved.
That's probably the best outcome an engineering solution can have.
When a process becomes so smooth that people forget it was ever difficult.