DataUp
Case study of engineering work at DataUp, featuring an AI Agents Management Dashboard and core CVAT modifications for Arabic RTL annotation pipelines.
Company Context & Mandate
DataUp is a data infrastructure and AI data collection company specializing in high-quality training datasets, annotation pipelines, and agent management for AI startups and research institutions across North Africa and the Middle East.
I joined DataUp as a Full-Stack Software Developer to strengthen their core engineering platform. My mandate focused on two key technical objectives:
- Designing and building a centralized AI Agents Management Dashboard to orchestrate, monitor, and evaluate autonomous data collection jobs in real time.
- Resolving severe rendering and data-processing bottlenecks in CVAT (Computer Vision Annotation Tool) to support complex Arabic right-to-left (RTL) text extraction, document parsing, and bounding-box annotations.
What I Built
1. AI Agents Management Dashboard
Architected and built a full-stack operational control room that allowed data teams and clients to deploy, monitor, and debug autonomous web-crawling and data-collection agents:
- Real-Time Agent Telemetry: Managed live status streams, task queues, memory logs, and error reporting across dozens of concurrent scraper and agent tasks.
- Granular Task Control: Implemented interactive controls to pause, resume, re-point, or adjust rate-limiting parameters on active agents without requiring terminal access.
- Data Quality Verification: Built pipeline interfaces for annotators to inspect raw scraped payloads before passing them to validation queues.
2. CVAT Core Modification for Arabic Right-to-Left (RTL) Support
CVAT's open-source frontend and backend had fundamental bugs when handling Arabic text annotation for OCR and document vision models:
- The Technical Challenge: Bidirectional (BiDi) text algorithms in standard HTML Canvas elements caused Arabic characters to render reversed, disconnected, or visually misaligned with bounding-box labels. Additionally, string manipulation in the backend API stripped diacritics and corrupted character positions during bounding-box spatial indexing.
- The Solution: Engineered a core open-source modification patch. Modified CVAT's HTML5 Canvas rendering engine using custom BiDi re-ordering logic (
bidi-jsintegration) to ensure Arabic script maintained proper ligature connection and right-to-left alignment inside active canvas bounding boxes. - Backend Normalization: Extended CVAT's Django backend models and serializers with UTF-8 Unicode normalization logic, ensuring spatial text offsets matched DB records exactly during export.
[ Canvas Interaction ] ──> ( BiDi Re-ordering Engine ) ──> [ Correct RTL Script Render ]
│
▼
[ Django Serializers ] ──> ( UTF-8 Unicode Normalizer ) ──> [ Accurate Database Offsets ]
Scope & Responsibilities
-
Full-Stack Architecture
Developed responsive UI screens using React and TypeScript while building secure backend REST API endpoints in Django REST Framework.
-
Annotation Pipeline Optimization
Refactored database queries (PostgreSQL/MySQL) to accelerate heavy JSON payload ingestion for multi-thousand image annotation tasks.
-
Cross-Team Collaboration
Worked alongside ML engineers, data operations leads, and annotator teams to identify UI friction points and optimize daily data throughput.
Impact & Results
Increased Arabic document annotation speed by ~40% by eliminating manual text workarounds previously required for broken RTL canvas labels.
Reduced agent error diagnostics time from hours of log-parsing down to minutes through the centralized management dashboard.
Successfully processed and exported thousands of high-precision Arabic OCR and document visual datasets for client AI model training.
Tech Stack Deep Dive
- CVAT Internals
- Deep engineering within CVAT's Python/Django backend and React/TypeScript HTML5 Canvas frontend.
- Django & Django REST Framework
- Core backend engine for user management, task orchestration APIs, and annotation data models.
- React & TypeScript
- Frontend stack for building responsive dashboard components, real-time status feeds, and custom canvas overlays.
- PostgreSQL
- Primary relational database handling spatial annotation offsets, project metadata, and audit logs.
Links & Resources
Lessons Learned
- 01
Open-Source Customization Requires Core Comprehension
Patching a massive tool like CVAT requires reading the codebase line-by-line rather than applying surface hacks. Fixing canvas rendering at the root saved weeks of downstream patches.
- 02
Internal Tools Accelerate Operational Output
Building dedicated management interfaces for agents directly impacts business velocity — giving non-technical operators control over complex infrastructure.
- 03
What I'd Do Differently
I would set up automated visual regression tests specifically for bidirectional language rendering earlier in the deployment pipeline to catch edge cases in rare Unicode characters.