When a client first brought us a custom chat export, we utilized custom scrips and internal processes t to convert undocumented XML data into fully functional RSMF files for review in Everlaw. That success proved feasibility. But the real test came later.
The Real Challenge Emerges
Months after the pilot, the client returned—this time with the full dataset.
- 17+ million individual chat messages
- Hundreds of embedded ZIP archives
- Multiple attachments, compounding document volume and complexity
This was no longer a quick win. It was an enterprise-scale project.
What Changed? Everything.
The volume and complexity of the data far surpassed the capabilities of standard desktop solutions. Additionally, rebuilding the entire processing workflow from scratch—including the generation of RSMF files—would have been unreasonably time-consuming.
As data volume and complexity continued to grow, the original approach became unsustainable. Increasing processing times and limited scalability ultimately necessitated a change in strategy.
Enter ModernETL.
Rapid Innovation with ModernETL’s “Custom Data” Module
Fortunately, Modern ETL’s high-performance parallel processing capabilities enabled us to handle individual data sets in hours rather than days. Its flexible architecture and custom data importer required only minimal customization. After a one-time initial parsing of the source data, the solution managed the remainder of the workflow—delivering an immediate and efficient turnaround.
Just as this new data set landed, our development team had finished testing the “Custom Data” module in ModernETL—a feature purpose-built for scenarios like this. It allowed our team to take bulk XML output, and push it directly into our ingestion pipeline.
- Scripted Conversion: Custom script transformed XML data into structured, easily ingestible formats optimized for scale.
- Mass Extraction: Sandline recovered and processed all archive files, including corrupted ZIPs, ensuring complete data capture.
- Seamless Transition: Workflow was migrated from desktop tools to ModernETL, significantly reducing overhead and increasing throughput.
All Hands on Deck
With a tight timeline and a massive scope, execution depended on team-wide coordination:
- Development accelerated testing using real client data
- Project Management & Hosting teams prepared rolling data loads into Everlaw
- Forensics & Data Operations collaborated on error handling and integrity validation
First batches were ready within days, and data is now being delivered on a rolling basis, ensuring the client can begin review while ingestion continues.
Why This Matters
This wasn’t just a scale-up. It was a real-world stress test of both our technology and our team.
- Scalability: Proof that ModernETL’s architecture can support millions of messages and dynamic ingestion workflows
- Speed to Review: No delays for development cycles—production began during testing
- Future-Proofing: The process is now repeatable for any custom chat export, no matter how niche
From Experiment to Enterprise Solution
This story started with a question: “Can Sandline process Symphony chat data?”
Now, it’s a scalable, defensible, and tested solution—live in production.
This is what agility looks like at scale.
This is what working with Sandline means.
