Much hype is on about Big Data and artificial intelligence applications and the focus is definitely upon machine learning to big data. Focus is on packaged solutions.
For the last few years, there has been endless talk about big data, led by Hadoop and now Spark. The next round of hype will no doubt be about applying machine learning to big data, but that is primarily to sell AI and analytics to people.
In reality, the big data era is indeed rapidly coming to a close. In fact, one can think of big data artificial intelligence has being quite significant in applications. One has come across media reports of big data pullbacks, which tend to put on a person in the trough of disillusionment in Gartner’s famous hype cycle.
One now comes across a situation whereby the big data “ends” and the actual application of the technology does begin.
For the industry, this does imply there will be fewer “let’s roll out the platform and see what happens” projects. The decisions makers are indeed are in for a more rational approach, as they should, and even start with a business problem initially. For example, big data artificial intelligence needs to be taken note of. In fact, even the platform companies need to talk about “solutions.”
Standard solutions for actual problems
The next big step is indeed analyzing problems, finding patterns, and also indeed creating packaged solutions to those problems.
One has already in fact observed seen this in the finance industry with the latest generation of distributed fraud detection packages that have been wrapped up as well as ready to go. Fraud detection software is in fact not so new, but on the other hand, distributing it at Hadoop and/or cloud scale is rather fresh. No doubt, finance is happening faster, but so is a fraud. For years, there has indeed been a missile gap — and the industry was rather losing out. Now they are rather fighting back, and Hadoop, Spark, and other modern tools are considered to be the firepower behind a new Custom-built solution thus making use of the next wave of technology that is not sufficient. Fraud detection for credit cards is in fact much different than for invoicing, insurance, or other common business applications. The next step is not to write for merely super- specialized apps for very specific industries, but rather to identify the “distributed big data patterns” in order to solve common problems that exist across lines of business.
One needs to take note of the fact that building custom solutions where everyone does manage to solve similar problems in different ways will rather persist for a while. But the future does involve efforts in finding commonality, developing patterns, and also spreading that across lines of business — that is, to make use of this new technology of massive distribution as well as cost-effective scale and apply it without blinders on. In the end, we do customize it and make use of the right terms and add the twists, but indeed designing pluggable algorithms in software that need not be written over and over again is what one is meant to gain expertise.
One has observed this before and decades ago, accounting software was considered to indeed be a hot topic. While one is able to find specialized accounting software for specific businesses, most of the big companies to make use of using a prepackaged solution that is customized to some degree or has a plug-in specific to the industry in question. It is rarely occurring that a skilled CIO or CTO is expected to write an accounting package for a line of business, apart from specific to the company.
The next major step to go ahead is focusing on “data-driven” and also making use of “machine learning” through a series of software package acquisitions and also trivial integration.
