Ironsight Data / Detailed showcase
Content ingestion & indexing
A backend and automation stack for collecting media, organizing its metadata, and tracking processing jobs. I built ingestion, duplicate detection, and workflow APIs so the content remains manageable as the collection grows.
Sole builder · Backend & workflow engineering
The problem
Growing content collections need consistent metadata, repeatable processing, and a way to detect duplicates and failures.
What I built
Built asynchronous collection and media download logic, job status APIs, PostgreSQL records, Redis-backed duplicate detection, and n8n orchestration. Retry and rate-limit handling support interrupted work. The containerized stack separates application, database, cache, and workflow responsibilities.
The result
Created a modular ingestion and indexing system with explicit processing state and reusable integration boundaries. Examples focus on architecture, with source-channel and personal data excluded.
Architecture / Simplified overview
How the pieces connect
- 01Source media
- 02Ingestion job
- 03Duplicate check
- 04Metadata index
- 05Workflow API