A client needed to defensibly process and prepare a large dataset from a proprietary internal software platform for legal review. With no existing parsing tools available and a complex mix of structured records, embedded chats, direct messages, and externally stored attachments, the client required a custom-built processing solution that could deliver review-ready output for Everlaw—fast.
The client’s dataset originated from a proprietary internal platform with no established processing workflows or off-the-shelf parsing solutions. The data contained a mix of:
- Specialized structured records
- Chat threads tied to those records
- Direct user-to-user messages
- Tens of thousands of linked attachments
In total, the dataset included approximately 700,000 items, including hundreds of thousands of chat messages and tens of thousands of attachments.
A major complication was that attachments were not stored directly in the platform. Instead, they existed as external hyperlinks, requiring the team to download, validate, and correctly map each attachment back to its parent record.
Further, the export process itself was challenging. The client needed to pull raw data from a database in multiple formats (CSV, XLSX, and JSON) and repeatedly re-export as requirements evolved and issues were uncovered.
All data ultimately needed to be delivered in a format compatible with Everlaw, including:
- Chat conversion into RSMF
- Preserved parent-child relationships
Accurate relational fields and metadata structure for efficient review
Our team partnered with the client from the outset, providing hands-on consultation throughout the export and processing process. As the client’s exports evolved, we continuously validated incoming data, flagged errors early, and helped prevent issues that would have surfaced later during review of ingestion.
We developed a custom processing pipeline from scratch to handle:
- Multi-format normalization across CSV, XLSX, and JSON
- Metadata extraction and validation
- Automated downloading and reconciliation of externally linked attachments
- Chat threading and conversion into RSMF
- Preservation of parent-child relationships using relational fields
- Structured delivery optimized for smooth ingestion into Everlaw
Although the overall engagement spanned approximately two weeks due to multiple export revisions, the final deliverable was completed in less than one week once the final dataset was received, reflecting rapid turnaround once the data was stabilized.
The final dataset—approximately 700,000 items—was delivered on target and successfully ingested into Everlaw without disruption. Throughout the engagement, the client highlighted the team’s responsiveness and proactive guidance, particularly during iterative refinements.
