The Engine Behind AI: Why Data Engineering is in Demand

As our world becomes more reliant on data, the need for skilled professionals who can manage, process, and prepare that data has grown significantly. One of the most important roles to emerge in this space is the data engineer, who builds the foundation for all data-driven insights.

But what does a data engineer’s career path look like? And how does someone interested in the field get started?

In our latest episode of Stories from the Hackery, we sat down with long-time Nashville Software School (NSS) friend and supporter, Jason Turan, to discuss the growing demand for data engineers. As one of the first hiring managers to tell us we should be training for these roles, Jason shares his perspective on why this career is so essential, especially in a data-rich city like Nashville.

A Partner in the Nashville Tech Community

Jason Turan has been a part of the NSS community for years, first getting involved around 2016 through the Nashville Technology Council. As a hiring manager in the healthcare and finance industries, he has volunteered for panel discussions, mock interviews, and has provided valuable feedback on our curriculum.

His belief in the potential of NSS graduates is clear. By the time he left his previous role at Healthcare Bluebook, about 70% of the junior data hires on his team came from Nashville Software School. His experience sourcing and developing talent gives him a unique view of the skills that new data engineers need to succeed.

The “Connective Tissue” of Data

So, what exactly does a data engineer do? Jason offers a clear and simple analogy. He describes data engineers as the “connective tissue between data producers and data consumers.”

He explains that data scientists and analysts often spend up to 80% of their time just cleaning and preparing data before they can even begin their analysis. A good data engineer flips that ratio. By building reliable systems that deliver clean, high-quality data, they allow analysts and scientists to spend their time on what they do best: finding insights. This makes the entire organization more efficient and effective.

Why the Demand for Data Engineers is Growing

While the need for data engineering has been steady for years, the recent rise of generative AI has acted as an accelerant for demand. As Jason points out, the key to getting high-quality output from any AI model is having high-quality data to begin with. The age-old mantra of “garbage in, garbage out” still applies today.

This trend is especially true in Nashville. With over 300 healthcare companies here, the city has a unique need for professionals who can pull data from complex, siloed systems like electronic health records and claims databases. This challenge could be why Nashville has more data engineering roles on a per capita basis than similarly sized cities.

How to Get Started in Data Engineering

For those considering a career in data engineering, Jason believes there has never been a better time to get into the field. He notes that the barriers to entry have never been lower, with open-source tools like DuckDB and container platforms like Docker making it possible for anyone to start learning and building projects on their own computer.

His advice for newcomers is to focus on developing a T-shaped skillset, with a deep understanding of SQL as the most critical skill. Rather than getting overwhelmed by the need to learn every new technology, mastering the fundamentals is the best way to start.

Ultimately, his best advice is to start building. "Go out and do the thing," Jason encourages. "Go out and just make mistakes and don't worry about somebody telling you the way that you're analyzing something or building something is wrong." Experimenting, building projects, and developing your curiosity are the most important first steps.

To hear more of Jason's insights and his advice for anyone considering a career in data engineering, listen to the full conversation on the Stories from the Hackery podcast.

Listen on SoundCloud or watch on YouTube.