13 New Trends in Big Data and Data Science


By Vincent Granville, data science executive

Based on requests from clients – vendors of data processing platforms and products – as well as trends in popular blogs,  job postings, and my own reading, here are 13 new trends in big data and data science recently gaining strong traction (items beyond #13 were recently added):

  1. The rise of data plumbing, to make big data run smoothly, safely, reliably, and fast through all “data pipes” (Internet, Intranet, in-memory, local servers, cloud, Hadoop clusters etc.), optimizing redundancy, load balance, data caching, data storage, data compression, signal extraction, data summarization and more. We bought the domain name DataPlumbing.com last week.
  2. The rise of the data plumber, system architect, and system analyst (a new breed of engineers and data scientists), a direct result of the rise of data plumbing
  3. Use of data science in unusual fields such as astrophysics, and the other way around (data science integrating techniques from these fields)
  4. The death of the fake data scientist
  5. The rise of the right-sized data (as oppose to big data). Other keywords related to this trend is “light analytics”, big data diet”, “data outsourcing”, the re-birth of “small data”. Not that big data is going away, it is indeed getting bigger every second, but many businesses are trying to leverage an increasingly smaller portion of it, rather than being lost in a (costly) ocean of unexploited data.
  6. Putting more intelligence (sometimes called AI or deep learning) into rudimentary big data applications (currently lacking any true statistical science) such as recommendation engines, crowdsourcing or collaborative filtering. Purpose: detecting and eliminating spam, fake profiles, fake traffic, propaganda, attacks, scams, bad recommendations and other abuses, as early as possible.
  7. Increased awareness of data security and protection, against computer or business hackers.
  8. The rise of mobile data exploitation. For instance processing billions of text messages to detect the spread of a disease or other global risks, to help design alarm systems or market the right product in real-time (via opt-in, user-customized text messages) to a walking customer in a shopping mall. Not sure that even the NSA is capable of doing it as of today. The issue is more about capturing and reacting to the right signal, rather than absorbing/digesting big data. Another trend is optimization of revenue from mobile apps, leveraging mobile app dashboards.

Read the source article at Data Science Central.