How Spotify use Big Data

Jan 14, 2020

Spotify is the largest on-demand music service in the world. It has a reputation of pushing technological limits and using big data and machine learning to drive success. The digital music company with more than 100 million users, have been busy this year enhancing their service through several acquisitions.

When a service have millions of people listening to music almost every second of the day, it means they will have access to an astonishing quantity of intel, including what songs are played most, where listeners are streaming from and even what devices are being used to access the service. Hence, there is no doubt Spotify is a data-driven company and it uses the data in every part of the organisation to drive decisions. As the service continues to acquire data points, it is using that information to train the algorithms and machines to listen to music and extrapolate insights that impact its business and the experience of listeners.

One example of how Spotify utilises their users Data to improve their listeners experience is through the popular feature, Spotify “Wrapped”.  The anticipation begins to grow for Spotify users when December arrives, as they wait for the music streaming service to deliver the Spotify “Wrapped”, which reveals the avid music fans’ favourite and most listened to songs/albums from throughout the year.

Spotify is one of many companies using customer and outside information to improve experiences for their customers. For their “Discovery Weekly” playlist feature, wherein every user gets a personalised playlist weekly of music that they have not heard before on the service, but that will be something the listener is expected to enjoy. Spotify is only using a few of the many possible techniques available to make recommendations for its users. With improving technologies and analysis techniques, data-driven customer experience will become the norm across most, if not all, companies in the years to come.

Data Set Online Course

Dec 11, 2019

The Data Set Online Course is the final output produced by the project partners. The objective of the course is to provide a multilingual, interactive learning course enabling direct access to educational resources and skills development activities for entrepreneurs wishing to improve their skills in data generation, management and analysis.

The Online Course is largely aimed at SMEs and business owners wishing to improve their knowledge, skills and confidence in order to grow their business and become more innovative entrepreneurs. For too long, big data has been perceived as a subject only for large corporations, but in recent years the cost of data analysis has dropped considerably and it is an opportune time for SMEs to learn how to use data to stay ahead of the curve. While there are many generic digital competences courses for entrepreneurs, this is the first online course dedicated especially to smart data skills.

The Online Course Modules Include:

  • Module 1-Introduction to Big Data

The objective of this module is to provide an overview of the basic information on big data.

  • Module 2-Improve your business with your own existing data

The objective of this module is to gain an overview of how to use the data you already have available in order to improve your business.

  • Module 3-Improve your business using external data

The objective of this module is to gain an overview of how to utilise data available outside of your company to improve your business.

  • Module 4-Legislation

The objective of this module is to gain an overview of the ethics surrounding big data and the legislation that governs it.

  • Module 5-The Future of big data

The objective of this module is to provide an overview of what the future impacts of big data are likely to be.


The online course facilitates flexible, user-centric learning meaning users can jump straight to the modules they find relevant. We created a simple-to-use, mobile friendly course meaning that their learning is portable and accessible for anywhere, to coincide with business owners’ demanding lifestyles.

The Online Course is coming soon, so keep your eyes peeled!



Nov 1, 2019

The GENERATION DATA Toolkit is the first output published by the project consortium. Its objective is to raise awareness about the increasing importance of smart data skills for current and future higher education students and provide knowledge about how to improve the teaching of smart data management.

The Toolkit is primarily addressed to HE teachers and lecturers who teach management-related subjects but can be used also for technical disciplines having a management-related component. At the same time, it represent a useful tool for policy-makers and other stakeholders involved or interested in entrepreneurship education.

The document includes:

  • The results of the data skills survey carried out by the partners in the initial project phase to find out which are the current needed skills and deficits of teachers and students when it comes to data skills
  • A review of the political and regulatory context governing data skills education at both EU and national/regional level
  • An introduction to strategies for teaching data skills to students and to early stage entrepreneurs, including best practices and testimonies

The Toolkit wants to be an easy-to-use and informative document, useful to support teachers and lecturers who have little time to dedicate to professional training but are nevertheless interested in deepening their understanding of the smart data skills world and to learn how to improve their teaching of smart data use and management.

Built upon the EntreComp, DigComp and DigCompOrg frameworks, the Toolkit has been conceived keeping in mind its transferability potential: although the focus is mainly on data skills teaching in higher education, the material can be useful also for teachers and trainers working in other education sectors and in digital education in general.

The GENERATION DATA Toolkit has been developed as an innovative open resource and as such is downloadable for free from the project website.

Government Collected Data and It’s Uses

Oct 24, 2019

Big data is a general term for the massive amount of digital information being collected from all sorts of sources collected into databases using various techniques. 2.5 quintillion bytes of data added each day to these databases with the majority of the world’s information collected in the last five years and with only 10% of that data being refined. 90% of the data is raw and unstructured just sitting there waiting to be analysed. This mammoth amount of big data, specifically gathered by means of online methods, mined by the government harbours the ability for businesses to use for a strategic or financial benefit. The government has begun to derive insight to help support business decision making in real-time from multiple sources, including the web, biological and industrial sensors, video, email, and social mediums. 

Many white papers, journals, articles, and business reports have proposed ways governments can use big data to help them serve their citizens by providing businesses with an information reservoir that can be accessed with the desired outcome that businesses can profit from the information and stimulate job creation. This would be a motivation for the government as lower unemployment rates would mean a greater pool of people and profitable businesses to tax ensuring a healthy revenue stream to maintain social services. Like with any new uncharted advantage; however, there are many things to consider such as security. Any form of personal information is subject to classification and risk to a person’s privacy. With greater risk that the information gathered can be used to impersonate an individual more complex methods of new technology will need to be used for security purposes to protect the big data.

It can be argued that highly sensitive personal data shouldn’t ever be made available for a business; however, with proper precautions and fees in place, it would seem like a wasted opportunity not to. 



Business and  Government Compared

The private sector is well known for making decisive decisions especially if it will maximise the profits in the short-term. Whereas the public sector is known to take time making decisions that will affect its citizens in the long term. For that reason to reduce risk and increase the efficiency and effectiveness of a decision. It follows that big-data applications likewise differ between public and private sectors.

In business, the main goal is to earn profits by providing goods and services to satisfied customers. The government’s main goal is to maintain the welfare civil order of its citizens through sustainable services. Although the primary missions of businesses and governments are not supposed to be in direct conflict with one another, data that is gathered by the government, for public sector benefits, can be a for the private sector business strategy. 

Big-Data Applications in Northern Ireland

Northern Ireland sadly has an imbalance of public versus private sector work where it currently only sits at 40% of the sum total. This large public sector is subsidised by the taxes collected by the rest of the United Kingdom. It is therefore in Northern Ireland’s best interest to encourage private sector growth as much as possible in the form of government-funded initiatives.

These initiatives realise market research is critical to a companies success. They can help small and large businesses to plan and execute marketing activities effectively by providing in-depth information to help break into new markets. An example of a government initiative with the express intent on gathering data would be Invest NI in Belfast. Their headquarters has extensive market research and worldwide company directories that can be used for free to a point. Their range of business information includes details of trade fairs, guidance on import/export procedures, funding sources and legal agreements, companies by sectors, their owners and telephone numbers.


E-zine Issue 2

Sep 30, 2019

Data Set project partner meeting no.2 – Spain

Sep 18, 2019
Partners pose with the DataSet logo

From the 30th to the 31st of May, 2019, the DataSET project consortium paid a visit to the birthplace of Cervantes, a World Heritage Site which transcends the 16th century spirit through its early Renaissance architecture – Alcala de Henares, Spain. With a warm welcome from the representatives of Universidad de Alcala, the consortium held the second transnational partner meeting to share the progress and discuss the future of the project.

The two-day programme provided the participants with the opportunity to discuss the status-quo of the development of the intellectual outputs, dissemination efforts and plans for the sustainability of the project.

Output 1 – VET Guide to Data Skills Development

We are delighted to announce that the first output of the project – the VET Guide to Data Skills Development has been finalized. The representative from the partner, coordinating the creation of the guide (Sonia Naiba, Momentum) presented the final version of the guide to the consortium. The guide aims to raise the awareness of the value of the data skills for current and future entrepreneurs and to analyze what contemporary data skills are already known to the entrepreneurs and business advisors. In addition, the guide summarizes the strategies for teaching data skills to entrepreneurs, including the best practices.

Specifically, the guide gives the overview of the following:

  1. i) The results of a data skills survey, outlining the current skills and skills deficits of business trainers and advisors in participating countries;
  2. ii) Review of the policy environment regarding data skills for entrepreneurs and data skills education, at both EU and national and regional level;

iii) An introduction to strategies for teaching data skills to entrepreneurs, including best practice examples and testimonies.

The guide is in open access and can be downloaded here

Output 2 – DataSet Open Education Resources

The DataSet Open Education Resources will comprise of a curriculum, trainer’s guide and a suite of interactive online learning materials which enable teachers and trainers to enhance entrepreneur’s data skills in classrooms and small group training. The Leading partner of this intellectual output, Universidad de Alcala and its representative (Miguel Ángel Sicilia Urbán, a Data Scientist himself), presented the draft methodology for the data skills training model and led an extensive discussion on the DataSET curriculum with the consortium. Resulting in almost finalized draft, all the partners are to pilot test the curriculum in small group session with business advisors and entrepreneurs in fall 2019.

The training model and its accompanying resources will be testes during the Train the Trainers Learning Activity in Denmark to be held in April 2020.

Output 3 – DataSet Online Course

In the meeting the partners discussed the preliminary plans of translating the open educational resources, created as a part of the previous DataSet output, into an online interactive learning course for the entrepreneurs at all stages of their business development. Self-paced and open to virtually anybody, the course will open the opportunities for popularizing the acquisition of data skills among wider European (and international) population.

In addition to the talks about the intellectual outputs, the partners have briefly discussed the future arrangements for the learning week, dissemination and exploitation plans and administrative issues regarding the project’s implementation phase.

Overall, the meeting was a success in terms of its outcomes, supported by the generous hospitality of the host partner. The partners enjoyed the traditional central Spanish cuisine and its famous tapas and a view of a historic city center of Alcala de Henares.  The next partner meeting is planned to take place in Leitrim, Ireland on 10-11 October, 2019.

EU Data Policy-Making: from Open Access to Developing Data Economy

Aug 28, 2019

The development of computer technology and digitalization made it possible to mine and store a massive amount of data. It allows businesses to identify new trends which can be used to make better decisions and seize new opportunities.

Big data has become a powerful driver for economic growth, competitiveness, innovation, job creation and societal progress. The EU aims to reap the full benefits of “big data fever” and maintain steady growth of Digital Single Market. According to the recent study (, the value of EU big data was more than €376 billion in 2018, accounting for 2.6 % of the EU GDP. With time-bound policy measures and favorable legal conditions, the value of the EU data economy can more than double by 2025 and represent more than 6 % of the overall EU GDP ( Big data brings new opportunities and the EU is acting fast to bring predictable rules of games to the table.

 The need for decisive steps and concrete actions firstly emerged in 2003 when the re-use of open public sector information (PSI) became legal for commercial and other purposes ( It established a minimum set of rules and the beginning of ‘open data’ era. Transparency and fair competition became key components of the ‘PSI Directive’ (Directive 2003/98/EC) The focus of this initiative was primarily economic and it had a great impact on the further development of new services based on novel ways to combine and make use of public sector information (PSI). Open data policy came into force.

The next big stepping-stone was the launch of the EU strategy ‘Towards a thriving data-driven economy’ in July, 2014 ( It was acknowledged that “data is at the centre of the future knowledge economy and society” and an action plan was adopted. It was based on the following pillars: community building and developing framework conditions for the single EU big data market. Fostering open data policies, E-infrastructure, Internet of Things (IoT) and personal data protection issues were covered in the new EU strategy on the data-driven economy (2014). A special attention was paid to the development of relevant data skills and infrastructures to the benefit of SMEs (

Developing a common European data space and economy, the EU faced three main obstacles to data mobility within the EU in 2017-2018. Among them, unjustified restrictions to free flow of non-personal data from Member States, legal uncertainties and lack of trust from main actors. According to the survey (2014) ( ), 38 % of SMEs in the EU-28 lacked trust in data mobility due to security risks. As a result, the Commission organized a number of public consultations with stakeholders to address the existing issues. As a result, the EU adopted a set of measures (outlined in Communication on “Building a European data economy”) to make a free flow of non-personal data across borders available and regulate new data technologies in terms of data access, portability and liability. Data privacy regulation was also put on the policy agenda, resulting in the adoption of GDPR and ePrivacy legislation. These measures were designed to enhance digital trust and protect the data privacy for EU citizens (

The big data agenda transformed from an open access data issue to building a common European data economy throughout the past 15 years. Even though the efforts of EU policy-makers were intensified, the central concern has been still the same: the EU can lose its competitive advantage of a Digital Single Market if it does not build a data-friendly regulatory framework. European SMEs have a risk of losing the competition in global markets if they have limited access to data analytics and data. Financing and the relevan data skills are particularly big difficulties to SMEs that do not have enough resources to invest in the data infrastructure and analytical tools. The EU should take a more proactive approach to support digital transition and further develop the European Data Economy.

Image credit:

Written by Aleksei Simonov, UIIN

The New VET Guide to Data Skills Development – Momentum

Aug 8, 2019


In today’s digital, connected world, the savviness with which entrepreneurs employ information and communication technologies is essential to competitiveness. However, while digital communication skills have improved across the population generally, the ability to leverage information, especially data, is still underdeveloped. This is a lost opportunity: the volume of data that business owners have access to has grown exponentially and if “big” data is turned into actionable “smart” data, it can drive productivity, innovation and growth.

The EU states that “data-driven business models are the engine of Europe’s growth, industrial transformation and job creation”, which is part of its commitment to the digitalization of the economy.

One of the benefits is that businesses responding to smart data can improve products and services, which would, in turn, generate economic growth while contributing to social progress. However, micro-enterprises and SMEs, which make up 99% of businesses, still lag in digital technologies. Micro-enterprises and SMEs must develop data skills or risk being uncompetitive, if the European economy is to flourish.

Nevertheless, there is an obstacle: today’s entrepreneurship teachers and trainers also face a data skills deficit. The majority entered the workforce before big data existed and there is currently no reliable source of training to help them boost their own skills. Prior to the start of the Data set project, East Belfast Enterprise conducted a small survey across 28 Local Enterprise Agencies in the Enterprise NI Network which found that “52% of business advisors said they were completely unaware of the range of data that is available and 70% rated their own knowledge of data skills as poor.”

About the VET Guide

The objective of the Guide is to raise awareness regarding the value of data skills for current and future entrepreneurs and increase knowledge of what contemporary data skills are and how they can be taught.

The Guide presents a comprehensive introduction to the role of data skills in VET and includes the results of a data skills survey, outlining the current skills and skills deficits of business trainers and advisors in participating countries, a  review of the policy environment regarding data skills for entrepreneurs and data skills education, at both EU and national and regional level and an introduction to strategies for teaching data skills to entrepreneurs, including best practice examples and testimonies.

Needs Analysis Assessment

The basis of the VET Guide is a needs assessment, which is a systematic process for determining and addressing needs, or “gaps” between current conditions and desired conditions or “wants” of a specific group. The chosen method for conducting the Data Skills Needs of Business Trainers and Advisors in Ireland, Northern Ireland/UK, Spain, Netherlands and Denmark was an Internet survey. This method was selected because it allows for a more diverse survey sample as survey link was widely shared online, it is a low-cost, fast and efficient method and the extensive networks of the partners allowed for a ready-made pool of participants.

The survey was made up of 12 short questions, it had a 100% completion rate and it was completed by 33 Business Advisors from 5 countries (Ireland, Northern Ireland/UK, Spain, Netherlands and Denmark).

Needs Analysis Survey Results

Data Skills proficiency is quite low among business advisors, with only 21% of those surveyed feeling their skills are proficient. The acquisition of Data Skills is of great importance to business advisors. 81% of those surveyed indicated that they would be interested in receiving/accessing free training and/or practical resources that they could use to teach entrepreneurs and SME owners about applicable data skills to their businesses.

Business Advisors today favour a Hands-On approach to providing business support therefore our data set materials should be very practical in nature and be solution oriented. Five key areas were identified where business advisors need upskilling with regard to digital skills and also 5 key areas which are particularly relevant for SME’s  – these are ranked in order of importance in the table below:

Data Skills for Business Advisors Data Skills for SME’s and Business Owners
Data/Information Analysis Application of Data to solve problems/inform business ideas
Reporting Skills Communication Skills
Application of Data to solve problems/inform business ideas Data/Information Analysis
Data Collection Creative Thinking


Technical/Digital Skills Technical/Digital Skills


The VET Guide also includes a section that goes in depth with regard to the Policy Environment regarding Data Skills for Entrepreneurs and Data Skills Education in the UK, Ireland, Spain, the Netherlands and Denmark, as well as a section dedicated to Strategies for Teaching Data Skills to Entrepreneurs.

The main goals of the VET Guide to Data Skills Development are to raise awareness of the value of data skills for business advisors and entrepreneurs and approaches for the delivery of data skills training and to lay the foundation for the data set Open Education Resources, which will consist of a curriculum, trainers’ guide and suite of interactive online learning materials which will enable teachers and trainers to enhance entrepreneurs’ data skills in classroom and small group training as well as for the data set Online Course, which will consist of a multilingual, interactive learning course in which entrepreneurs at all stages of entrepreneurial activity can learn more and put data skills into practice.

More detailed information regarding the findings of the online survey are available in the VET Guide, which can be accessed in its entirety on the project website.

Data Skills Development Initiatives

Jul 26, 2019

Benefits of Big Data for Business Organizations

There are many small and large business organizations that are making use of big data to derive a lot of amazing benefits. Below are a few examples of how gathering data and using already gathered data can help a business in different stages of its development.

Risk Analysis For Start-Ups

 Start-up companies owners have a lot of things to consider when they are building their business. Sometimes it can be all too easy to become laser-focused on just the day to day running of the business when there other important factors that should be considered if they are to reach set business goals. “Big Data” can easily help with a start-up business assess their social impacts, economic importance and their accomplishments.

Data gathering can be as simple as analyzing peer-reviewed journals, newspapers, social media feed and surveys. The information already gathered by these sources can allow a business to avoid common problems and save money.

Product or Service Re-Development

 Are you currently running a business? Then you may be surprised to learn that data collection, or “Big Data” can be of great importance to you. You may already know about the importance of feedback and be able to use it to benefit your business, but collecting the information is only one aspect of what you could be collecting to help your business. Big Data can help you figure out how customers perceive your product or service and in turn, you will be able to revamp or redevelop your product or service to improve its reputation. Designing the product around the needs and wants of your base clientele is a key to a successful business.

Product or Service Expansion

Consumers simply know what they want. They may not always know what they need, but they will always know what appeals to them and where their priorities lay. It is because of this understanding that they will do their own research before they commit to a product or service. They will read reviews and watch Youtube videos regarding the product and potential comparison products after maybe seeing an advert that piqued their interest.

With “big data”, different business can profile customers in a far-reaching manner. Business owners should be aware of these review and comparison sites to check the various desires as well as the preferences of their customers. Based on it, they can offer the products and services tailored to their customers. They can also do a cost analysis of the customer’s desires to see if a product or service is viable in the first place.

Targeted Marketing & Data Management

Once you have gathered your data in the form of customer feedback, emails, names, social media account insights and customer addresses; you may wish to employ computer-aided software to begin advanced information breakdowns for marketing purposes. For instance, you may wish to pinpoint a where on the map the majority of your clients come from, the gender split or simply which social media platform is predominantly being used by your clientele. All this information will allow you to create tailored marketing campaigns especially if your marketing budget is limited.

Staff members should be trained on how to gather data as a standard and about “big data” management, which is not as simple as you think and there can be large penalties associated with mismanagement of customer data. Where “big data” is sensitive to the identity of a client or customer, such as financial, medical or criminal records, data must be encrypted and protected from public access. By this, we mean to say that this sort of data should be kept offline and in no way accessible from a cloud source. Such breaches of privacy can be costly to any company and in some instances mark the end of business entirely due to the fines incurred.

Methods of Analysing Data

Jul 14, 2019

Implementing a business intelligence software in your company or organisation is more than simply about collecting additional data; it is about making this data work for you in the form of actionable information. The amount of data an organization can collect today from a variety of sources offers is staggering. The ability to see what’s behind the curtain, understand what campaigns or actions are working can help a business owner prepare for future trends.

However, without having a proper understanding of the data that is collected, all you those figures, numbers and statistical social insights become substance with no context.

It should be noted that there isn’t anyone correct method for analysing data. It is primarily dependent on the needs of a business and the data they aim to collect that will dictate which methods of analysis will best suit and even at that, techniques can be fluid to deliver the best results. In saying that, there are some tried and tested methodologies that are built into different software because they do work.

The first step in choosing the right data analysis technique for a data set begins with understanding what type of data it is. It can either be quantitative or qualitative data. Someone may ask what exactly is the difference between the two? Quantitative data deals mostly with the volume of information. It is the cold, hard facts that can only be brought in by the numbers. For example, the number of sales, click-through rates (CTR) on online marketing campaigns, return on investments (ROI) and other such measurable figures that can be scrutinised objectively.

Qualitative data is a little more ambiguous and nebulous, but no less valuable. It deals more with being subjective, needs a degree of introspection and very much open to interpretation. This sort of data would apply to things like customer reviews, surveys and even watching for customer and staff interactions with a product or service. It is about the quality of how a product or service is being perceived and this makes methods of analysing this sort of data a bit more difficult since it can often be less structured.

Below are three different methods that can be used to measure quantitative data and two that deal with qualitative data to hopefully might help a business begin to categorise and breakdown the data they may already hold.

Measuring Quantitative Data

Regression Analysis

In order to understand regression analysis fully, it is imperative to understand the following terms:

Dependent Variable: This is the main factor trying to understand or predicted.

Independent Variables: These are the factors that can be hypothesised have an impact on the dependent variable.

Whereas you can only have one dependent variable, you can have a myriad of independent variables. Regression analysis is the measurement of the independent variables to the dependent variable and when understanding the relationship to the dependent variable can be used to help a company make a prediction about future trends with a good degree of confidence. Regression analysis is a reliable method of identifying which variables have an impact on a topic of interest. The process of performing a regression allows you to confidently determine which factors matter most, which factors can be ignored, and how these factors influence each other.

Hypothesis Testing

Also known as “T Testing”, is used to infer the result of a hypothesis performed on sample data from a larger population. The test tells the analyst whether or not his primary hypothesis is true. For instance, a business owner may assume that more hours of work are equivalent to higher productivity. How exactly would they know this to be factually the case? Before implementing longer work hours, it’s important to ensure there’s a real connection to avoid spending money on wages or even whether the longer hours would even be received well by staff. A business decision made without knowing how it may affect staff may have the opposite effect where productivity may drop.

Monte Carlo Simulation

Monte Carlo simulation is a technique used to understand the impact of risk and uncertainty in financial, project management, cost, logistics and other forecasting models. A Monte Carlo simulator helps one visualize most or all of the potential outcomes to have a better idea regarding the risk of a decision. To test a hypothesis or scenario, a Monte Carlo simulation will use random numbers and data to stage a variety of possible outcomes to any situation based on any results. It will allow the analyst to understand what random variables can throw a monkey wrench in a company’s project or strategy.

Measuring Qualitative Data

Content Analysis

Content analysis is a research technique used to make replicable and valid inferences by interpreting and coding textual material. By systematically evaluating texts (e.g., documents, oral communication, and graphics), qualitative data can be converted into quantitative data.

Narrative Analysis- Narrative analysis is a genre of analytic frames whereby researchers interpret stories that are told within the context of research and/or are shared in everyday life. This might include interpreting how employees feel about their jobs, how customers perceive an organization, and how operational processes are viewed. It can be useful when contemplating changes to corporate culture or planning new marketing strategies.