Author: Ranjit Rajagopalan
Artificial Intelligence (AI) is a technology big data enables. Harnessing big data has resulted in transformational technologies, but none hotter than AI.
AI and its subset Machine Learning (ML) is driving the need for efficient big data processing. Clean data, which is representative of the population, is essential for AI engines to deliver on the competitive advantage promise. This may be a reason why data is considered the new oil.
Here I discuss the outcomes of the current AI revolution, how we have harnessed the power of big data and AI at CGI and my predictions for what’s coming next in this space.
The impact of AI on data gathering technologies
Continuing on with the data/oil analogy, if one imagines raw data as crude oil, then there is a need for a data ‘refinery’ as well. This need for refined data has powered some fascinating innovations in the field of data gathering.
In the new digital world, for example, we cannot use traditional databases for storage as the big data that gets generated is varied, voluminous, and coming at a speed that traditional databases struggle with.
This has led to innovative file-based distributed storage mechanisms that are not only resilient but also cost-effective. In-memory columnar database technologies are also able to alleviate some of these problems. Additionally, real-time ingestion technologies have been developed to consume this data without any loss.
Furthermore, with large data sets, we now have increased complexity of data retrieval. This has led to efficient data retrieval techniques based on patterns and trade-offs. Suffice to say that techniques such as Indexing Mechanisms and calibration of the precision/recall trade-off make this possible.
And finally, data needs to be imputed, cleansed and transformed for the AI engine to process. Feature selection algorithms, as well as open-source technologies providing efficient and innovative data transformation mechanisms, make this task feasible.
In essence, the AI revolution has triggered a massive improvement in data-gathering technologies.
Practical examples: My recent projects with CGI
At CGI, we’re working on a number of projects that harness the power of big data and AI. One such project I worked on recently involved incubating a product that harnesses real-time streaming data from our IoT data ingestion platform and provides innovative insights.
I also worked on an application which called a model based on historic sensor readings from multiple sensors (Temperature, Rainfall, Level), and accurately predicted reservoir levels days into the future.
And, along with our partners, we’ve created a joint prototype wherein video feeds from a dense urban interchange were analysed and feedback provided in real-time with the aim of improving citizen behaviour.
There’s more on the horizon, too. We’re in the process of building a leading I&O (infrastructure and operations) solution that will apply automation and AI to the core of IT operational management (ITOM). This approach will help our clients tackle the growing emerging challenges by simplifying and reducing the total cost of operations and helping accelerate key transformation and innovation initiatives.
The solution will transform how our clients see and manage their IT environments through:
- Observability – detecting anomalies in real-time, grouping alerts, smart alerting and providing 360-degree visualisation of IT and network infrastructure
- Automation – a more responsive, timely and accurate resolution approach that addresses IT issues, vulnerabilities, compliance and general IT operations
All of these projects have been incredibly exciting to work on, and they really highlight how we’re leveraging data and AI together to help drive innovation for our clients.
Future potential for AI and big data – what's next?
So, what’s next for AI and big data in the broader market? I am keenly tracking a few applications and considerations:
Sustainability: AI, when combined with telecommunication technology shifts such as 5G, will enable a much-needed drive towards sustainability, especially in urban areas.
Harnessing rich data from sources such as IoT environmental sensors, visual feeds, citizen mobile phones, and social media feeds result in applications that will improve the environment and the lives of citizens.
Personalisation: In order to stay competitive, there is a drive amongst organisations to understand their customers well. In the digital world, this personalised approach is a big challenge as it requires combining multiple data sources such as geolocation, browsing preferences, transactional patterns and customer demographics and psychographics.
AI is helping in this cause and we see this appearing in our daily digital interactions. Ultimately, digital channels will be able to provide hyper-personalised services that were not possible in the physical world.
Observability: While the predictive aspects of AI steal the limelight, I feel the power to live in the present can be further enhanced. Systems are becoming increasingly complex and the task of observing and monitoring the current state requires new innovative solutions. One only needs to imagine a self-driven car and the number of parameters that need to be observed in real-time to make fraction-of-second decisions.
You can learn more about the future of AI for our clients through our 2022 Client Global Insights.
When it comes to AI and big data, the future is bright. If you’re keen to learn more about the work we’re doing in this space, and where you might fit into the puzzle, take a look at the current career opportunities at CGI.