Organizations today recognize the imperative of using their data as a strategic asset to create competitive advantage.
However, organizational and technical friction points can slow down data-driven initiatives and create hurdles to unlock true data value.
ET Edge Insights in association with Altair, a global provider of computational science and AI, converging software and cloud, HPC, and data analytics solutions, held a roundtable in Bangalore on the topic of “Make data science a team sport.” 
The roundtable witnessed participation from Data Science leaders who shared their views around how they are connecting data with the business goals, the need to make data science teams work effectively, the rise of technical debt in software engineering, democratization of data and the emergence of Generative AI.
Marrying Data with Business Goals
Data is a key resource for businesses to gain insights and make decisions. And this can only happen when they can connect or link data with business objectives.
The speakers argued that data is a binary entity, without any context. The other challenge is to combine data, especially from alternative sources, while keeping a strong ethical base. Each data point has a different story, and the process of getting data can be hard. Also, managing and following data rules adds more difficulty.
Sometimes, teams have a narrow view, working alone and not cooperating well. While the ideal situation is to spend 80% of time on data exploration and only 20% on model creation, businesses cannot always do that and must find a balance between the two.
Data scientists should be able to test and check their ideas, but they should also know the best ethical limits. Teams with different skills are important for this process, as they bring different views that improve decision-making.
To match data with business objectives, it is important to ask the right questions to stakeholders, find the right problem, understand the data details, and use effective feature creation.
However, human bias is still a major issue. Biases are part of data itself, and past data tends to ignore some groups. The paradox of data science is that while models may show bias, the real cause is in the data. Luckily, there are ways to deal with data bias. Studying data behavior can help find biases and spending more time on data exploration can give useful insights. Developing a data-driven culture requires everyone to appreciate the significance of data and its impact on decision-making.
The Challenge of Technical Debt
A common obstacle for quickly adopting any new technology in production settings is technical debt (TD). It is not relevant what coding language, software package, computing method, host, or smart hybrid combination that enterprises use.
Old financial services firms have large scale operations and because of governance and compliance factors, dealing with technical debt is a slow process. Breaking any compliance can be expensive. It is hard to get rid of the outdated and low value tools and platforms and move data to newer ones.
Generative AI: Proceed with Caution
The pace of technological change brought forward with Generative AI will continue to accelerate, opening the door to new innovations, and business models.
However, the speakers suggested that enterprises should be careful about ethics, privacy and hallucinations when using this new technology. Open-source models will be difficult to use unless there are abilities to build, train and deploy them privately.
Using Generative AI to fight fraud can be a potential use-case. Another use case could be automatically generating workflow for data engineering, etc.
To sum up, combining data in a responsible and effective way is a complicated task, but by creating a cooperative environment, finding a balance between data exploration and model creation, recognizing biases, and improving data skills throughout the organization, businesses can use the power of data to drive innovation and success.
The speakers agreed that there is a clear need to align the data science team and the management with the single business goal to deliver full business value.
