My take on transforming data

Source: Dev.to

Introduction

When I started working in development back in 2011, life was simpler. It was just three of us building websites and managing data while still in university. We barely knew what we were doing, but competition was low, so we were fine.

The Changing Landscape

The shift came when we moved from custom‑tailored websites to building our own product. Customers arrived, bringing with them a maze of legacy systems—some even written in COBOL.

In the early years every new integration was paid work and relatively easy because we could create a dedicated module for each client.

Today the situation is different: the number of integrations has grown, and the needs are both internal and external, meaning not every integration is directly profitable.

Towards Reusable Solutions

My journey turned toward reusable modules, standardized APIs, and FTP workflows that use the same format as our APIs.

Early Experiments

• JSON schemas – I built a UI for them, but the project never excited me enough to continue in my free time.

Leveraging the AI Era

When AI tools became widely available, they gave me the ability to do more work in less time. This opened new possibilities and led me to a new idea: queries.

The Query‑Based Approach

The results have been promising. I’ve built:

• A command‑line interface
• An API server with automatic Swagger documentation
• A JavaScript/TypeScript library

The system can convert CSV, XML, JSON, and even more complex formats like EDIFACT and fixed‑length plain text.

Basic Example

from xml to json
transform
set id=CID
set number=number(progressive)

The snippet above transforms XML to JSON, extracting an id field from CID and a number field cast to a numeric type.

Playground

I’ve created an online playground that makes it easy to compose queries based on the data you have. As I use the tool, I keep adding new functions and adapters whenever a need arises.

Call for Discussion

I’m sure I’m not the only one who sees potential in this approach. I wanted to share it here in the hope of sparking interesting discussions.

If you find the project intriguing, feel free to reach out!