pyAggregate is a Python re-implementation of two SAS programs used to aggregate returned Sentinel quality assurance data and output it as parquet files. Rather than converting SAS code to Python, the program was rebuilt using Claude Code from a base set of requirements and JSON schema over the course of two days (not including human in the loop quality control steps). Implementing it from a Product Requirements Document and data model rather than converting the SAS code directly avoids baising the robot with a legacy codebase that baked in a lot of assumptions that simply don’t hold true in 2026.
pyAggregate
pyAggregate has two primary functions:
pyaggregate scan: Using cron the program polls a user defined directory or set of directories for data newly approved data to the coordinating center from data partners. The location of these data are saved to a database (sqlite3) for later querying.
pyaggregate run: Using cron once again, the program can be set to periodically query the database for the most recent returned data from each site. This data is then output in three flavors: stacked—does what it says on the box, masked—replaces data partner identifiers with surrogate ids, and rolled-up—drops the site identifier and summarizes across all sites.
Migrating this tool to Python has reduced the data footprint of the output to approximately 12% of the original, and enabled downstream business intelligence projects to take advantage of these data without requiring a manual step to convert the data from SAS to a format usable by our BI suite.