The Privacy Risk Most Data Scientists Are Missing

The Privacy Risk Most Data Scientists Are Missing

Facebook privacy issues

Data breaches are becoming increasingly common, and the risks of being involved in one are going up. A Ponemon Institute report (an IBM-backed think tank), found that the average cost of a data breach in 2018 was $148 per record, up nearly 5% from 2017.

Privacy regulations and compliance teams are using methods like masking and tokenization to protect their data — but these methods come at a cost.
Businesses often find that these solutions prevent data from being leveraged for analytics and on top of that, they also leave your data exposed.

Many data scientists and compliance departments protect and secure direct identifiers. They hide an individual’s name, or their social security number, and move on. The assumption is that by removing unique values from a user, the dataset has been de-identified. Unfortunately, that is not the case.

In 2010, Netflix announced a $1 million competition to whoever could build them the best movie-recommendation engine. To facilitate this, they released large volumes of subscriber data with redacted direct identifiers, so engineers could use Netflix’s actual data, without compromising consumer privacy. The available information included users’ age, gender, and zip code. However, when these indirect identifiers (also known as quasi-identifiers) were taken in combination, they could re-identify a user with over 90% accuracy. That’s exactly what happened, resulting in the exposure of millions of Netflix’s consumers. Within a few months, the competition had been called off, and a lawsuit was filed against Netflix.

When it comes to the risk exposure of indirect identifiers, it’s not a question of if, but a question of when. That’s a lesson companies have continuously found out the hard way. Marriott, the hotel chain, faced a data breach of 500 million consumer records and faced $72 million in damages due to a failure to protect indirect identifiers.

Businesses are faced with a dilemma. Do they redact all their data and leave it barren for analysis? Or do you leave indirect identifiers unprotected, and create an avenue for exposure that will lead to an eventual leak of your customers’ private data?

Either option causes problems. That can be changed!

That’s why we founded CryptoNumerics. Our software is able to autonomously classify your datasets into direct, indirect, sensitive, and insensitive identifiers, using AI. We then use cutting-edge data science technologies like differential privacy, k-anonymization, and secure multi-party computation to anonymize your data while preserving its analytical value. Your datasets are comprehensively protected and de-identified, while still being enabled for machine learning, and data analysis.

Data is the new oil. Artificial intelligence and machine learning represent the future of technology-value, and any company that does not keep up will be left behind and disrupted. Businesses cannot afford to leave data siloed, or uncollected.

Likewise, Data privacy is no longer an issue that can be ignored. Scandals like Cambridge Analytica, and policies like GDPR, prove that, but the industry is still not knowledgeable on key risks, like indirect identifiers. Companies that use their data irresponsibly will feel the damage, but those that don’t use their data at all will be left behind. Choose instead, not to fall into either category.

Join our newsletter



The Three P’s of Retail Success

The Three P’s of Retail Success

Facebook privacy issues

As a retailer, you have a limited view of your customer based on what you gather from your POS data and social media because you don’t know how customers are spending their money outside of your store. All this can be solved if you acquire access to a very useful piece of data-financial data.

By combining financial data from millions of customers with your POS data, you can achieve a solid 360-degree view of your customer based on their preferences and habits, and grow your ROI by running more targeted marketing strategies. Additionally, you can also outperform your competition by spotting trends and offering better deals.

Adding Financial Data to the Mix: The Benefits

With access to customer’s financial data, not only will you be able to make more informed business decisions, but think of all the efficiencies you would gain from additional customer knowledge and optimized marketing expenditure.

Personalization

    The amount of personalization possible with all this added financial data allows for stronger customer experience and retention. Talk about a mutual benefit!

There are two advantages when it comes to financial data working together with POS data to boost personalization: increased customer intimacy and increased customer loyalty. With customer intimacy, we are talking about being able to better anticipate customer needs by analyzing buying patterns and understanding shopper behavior. With customer loyalty, we are talking about how you can customize your offerings and deals according to a target group’s needs or even an individual’s needs, to ensure the customers feel heard and important.

Thus, personalization is always value-adding to both the customer and the company because it helps with bringing in value as well as customer retention.

Promotion

   With financial data in the mix, you are further able to maximize the quality of your marketing spend.

You could optimize your marketing expenditure by combining your sales data (for example, what was purchased at your store, when it was purchased and how much it was purchased for), and financial data (for example, how much was spent at your store versus another store within the same industry). Seeing these customer preferences allows you to obtain valuable insights which will help you make smarter product, pricing, and promotional decisions.

Another important aspect of leveraging financial data is knowing what percentage of your customer’s wallet is going towards you versus your competition. But wait, that’s not all! In terms of data insights, unlocking this greater potential will help your organization build more powerful models. Forecasting future sales and buying preferences will be much easier to detect with financial data.

Financial data can help to better direct your promotional efforts in terms of efficiency and information such as data insights, saving you both time and money.

Privacy

    Using privacy-protected financial data that is secure and compliant with all legislative regulation helps you be worry-free and avoid any problems or PR nightmares.

Luckily, there are companies out there that combine security and privacy to form an optimal solution to comply with regulation, ensure privacy and IP protection as well as secure the best possible ROI for your company. Additionally, their privacy and security methods are intact throughout the data pipeline, from acquisition all the way down to publishing, using access controls and cryptography.

Basically, combining all this financial data to your existing pool of data will help you increase local demand, optimize media spending and promotional activities, focus on customer experience, and it will also compliment your privacy compliance. Modelling all these levers can help you forecast future sales and growth as well, thus increasing performance.

Still not convinced? Let’s check out some large corporations that stand by this…

See How Walmart is Implementing this Solution

“Walmart uses big data to make the company’s operations more efficient and improve the lives of customers”

To power their goal to provide the best shopping experience possible, Walmart is maximizing their use of big data, to help reveal consumer patterns. Transactional, online and mobile data, all combine to help them serve the customer better, so they can keep coming back.

They use data mining to extrapolate trends from their POS data, to see what the customer buys, when they buy it, how they buy it (online or in store) and what they buy before or after a certain product. POS data allows the organization to see shopping patterns to determine how to display merchandise and stock shelves. Furthermore, they can send out personalized rollback deals and vouchers based on consumer spending habits. Not only do they use this data to create customer value, they also use it for staffing purposes. For example, to help lower the amount of time it takes to fill a prescription, Walmart looks at how many prescriptions are filled each day to determine staff scheduling and inventory (link).

Additionally, Walmart has created their own credit card which gives them firsthand knowledge on their customers. Using the expenses from the financial data of customers helps them gain a solidified understanding of consumer habits and preferences. This enables the company to anticipate demand for each product or service.

The outcome of using this big data includes improving store checkout procedures, managing supply chain and optimizing product assortment.

To Sum it Up

Without data, companies are not able to grow and digitally enhance themselves according to the needs of their target market. So being able to leverage data to its full ability is a competitive advantage on its own, especially with data being such a huge commodity today. Unlock greater potential with respect to increasing your customer value by expanding your access to the data available around you.

Join our newsletter



Announcing CN-Protect for Data Science

Announcing CN-Protect for Data Science

We are pleased to announce the launch of CN-Protect for Data Science

CryptoNumerics announces CN-Protect for Data Science, a Python library that applies insight-preserving data privacy protection, enabling data scientists to build better quality models on sensitive data.  

Toronto – April 24, 2019CryptoNumerics, a Toronto-based enterprise software company, announced the launch of CN-Protect for Data Science which enables data scientists to implement state-of-the-art privacy protection, such as differential privacy, directly into their data science stack while maintaining analytical value.

According to a 2017 Keggle study, two of the top 10 challenges that data scientists face at work are data inaccessibility and privacy regulations, such as GDPR, HIPAA, and CCPA.  Additionally, common privacy protection techniques, such as Data Masking, often decimate the analytical value of the data. CN-Protect for Data Science solves these issues by allowing data scientists to seamlessly privacy-protect datasets that retain their analytical value and can subsequently be used for statistical analysis and machine learning.

“Private information that is contained in data is preventing data scientists from obtaining insights that can help meet business goals.  They either cannot access the data at all or receive a low quality version which has had the private information removed.” Monica Holboke, Co-founder & CEO CryptoNumerics. “With CN-Protect for Data Science, data scientists can incorporate privacy protection in their workflow with ease and deliver more powerful models to their organization.”

CN-Protect for Data Science is a privacy-protection python library that works with Anaconda, Scikit and Jupyter Notebooks, smoothly integrating into the data scientist workflow.  Data scientists will be able to:

  • Create and apply customized privacy protection schemes, streamlining the compliance process.
  • Preserve analytical value for model building while ensuring privacy protection.
  • Implement differential privacy and other state-of-the-art privacy protection techniques using only a few lines of code.

CN-Protect for Data Science follows the successful launch of CN-Protect Desktop App in March. It is part of CryptoNumerics’ efforts to bring insight-preserving data privacy protection to data science platforms and data engineering pipelines while complying with GDPR, HIPAA, and CCPA. CN-Protect editions for SAS, R Studio, Amazon AWS, Microsoft Azure, and Google GCP are coming soon.  

Join our newsletter



Top 10 Challenges Data Scientists Face at Work

Top 10 Challenges Data Scientists Face at Work

We all have heard that “data is the new oil”. As with oil, data has to be transformed to be of real value to the society. The people in charge of this transformation are data professionals.

Data professionals are constantly trying to make sense of data by building models that can provide the insights necessary for organizations to grow and generate more value. However, these professionals face many challenges that prevent them from building powerful models.

In 2017, Kaggle did a study titled the “State of Data Science and Machine Learning”. One of the questions the survey asked was, “At work, which barriers or challenges have you faced this past year? (Select all that apply)”. Here are the top 10 results:

Here is a look at how often they encountered these problems:

 

 Most of the timeOftenSometimesRarely
Dirty Data43%40%16%1%
Lack of data science talent in the organization31%40%27%2%
Company politics / Lack of management/financial support for a data science team26%40%30%4%
Unavailability of/difficult access to data28%42%27%2%
The lack of a clear question to be answering or a clear direction to go in with the available data29%43%27%2%
Data Science results not used by business decision makers16%44%37%3%
Explaining data science to others19%41%36%3%
Privacy Issues25%36%34%5%
Lack of significant domain expert input22%46%29%3%
Organization is small and cannot afford a data science team37%36%24%3%

Data cleanliness is clearly a big issue, as data scientists spend 80% of their time cleaning data. However challenges like a lack of talent/expertise, company politics meaning results are not used, and data inaccessibility, are more difficult to solve as they require systemic changes within the organization.

To find how data professionals answered the other questions in the study, click here to visit Kaggle 2017 study.

Join our newsletter



Six Things to Look for in Privacy Protection Software

Six Things to Look for in Privacy Protection Software

This is the fourth blog in our Crash course in Privacy series

 

Enterprises want to:

  • Leverage their data assets
  • Comply with privacy regulations
  • Reduce the risk exposure of consumer information.

If the goal is to maintain data utility while protecting privacy here is a list of six key things you should consider in data privacy software:

1) Allows you to understand the privacy risk of your dataset

It is easy to think that by removing information like names and ID’s privacy risk is eliminated, however as shown by the Netflix case, there is a lot of additional information in a data set that can be used to re-identify someone, even when those fields have been removed. Therefore it is important to know what the probability of re-identification is of your dataset after you have applied privacy-protection. There are other lesser-known types of privacy risks that could matter to you such as membership disclosure and attribute disclosure.

The software you use should help you understand and manage these risks.

2) Enables you to understand information loss and maintain the analytical value

Every time you apply anonymization techniques to your dataset, the information is transformed. This transformation either redacts, generalizes or replaces the original data causing some information loss. Depending on what the data will be used for, you need to be able to understand the impact on your data quality. Your data quality could vary widely even with the same privacy risk, so knowing this makes a huge difference when using privacy-protected data for analytics.

Software that helps you understand the information loss and maintain analytical value after de-identification is critical.

3) Protects all attribute types

To achieve optimal privacy protection while balancing data quality, all data elements need to be classified appropriately. Incorrectly classifying a data element as an Identifier, Quasi-identifier, Sensitive, or Insensitive attribute could lead to insufficient privacy protection or excessive data quality loss.

The right privacy-protection software should support all four attribute types (identifier, Quasi, identifier, Sensitive, Insensitive) and allow you to customize the classification of your data elements based on your needs.

To learn more about the data attributes read Why privacy is important.

4) Supports a range of privacy techniques and is tunable

Each different privacy technique has pros and cons depending on what the data will be used for e.g Masking removes analytical value completely but is good for protection. You should look for software that supports a range of privacy protection techniques as well as tunable parameters for each of them to find the perfect balance for your needs.

5) Applies consistent privacy policies

Satisfying privacy regulations is a cumbersome and manual process. Being able to create privacy frameworks and share them across the organization for application purposes is key, so software that allows you and your team to apply consistent privacy policies is critical.

6) Your data stays where you can protect it

You are looking to privacy-protect your data, the software you use should work in the environment where you are already protecting your data. Using software that runs locally in your environment will remove an additional layer of risk.

 

The other blogs in the Crash course in Privacy series are:

Join our newsletter