Use AI to Create Synthetic Data from a DataFrame or CSV

In this tutorial, Alex walks you through how to use AI to create synthetic data from a DataFrame or CSV using Gretel.
Jupyter Notebook from the video on our GitHub: github.com/gre....
Run the notebook in your browser via Google Colab: colab.research....
-------
► Get started with Gretel: console.gretel...
► Join the Slack community: gretel.ai/slac...
Gretel is Privacy Engineering as a Service. We build privacy engineering tools delivered to you as APIs. Synthesize and transform data in minutes and build trust with your users and community.
Find us elsewhere on the internet
Website: gretel.ai
Twitter bit.ly/3BMGHRM
LinkedIn: bit.ly/3ycC5SM
GitHub: bit.ly/3zGpcAJ
Medium: / gretel
TIMESTAMPS
00:00 Tutorial: Use AI to Create Synthetic Data from a Dataframe or CSV
00:15 Create a New Project
00:40 Launch a Project to Create Synthetic Data from a CSV or Dataframe
01:08 Run the Python Script
01:29 Get Your Gretel API Key
04:05 What Is a Validator?
06:25 Review How Your Model Created Synthetic Dataset

[@muritalaadebayoisah9155]

Thank you for the wonderful presentation. It was very educative. Question1: Can gretel-synthetics be used to create a numerical data frame? Question2: What is the minimum number of rows that can be used for training the model in order to generate the synthetic data Question3: I tried to use it on a numerical dataset of shape (36,118) but was getting errors during training. I wanted to create a synthetic dataset of shape (200,18) from (36,18). Could you please guide me through. Thank you.

[@alexanderwatson1786]

Hey Muritala- thank you! Re: Question 1- yes, gretel-synthetics should be able to recreate any numeric or text-based data in a Dataframe Question 2- The more rows and diversity you have in the data, the better the model will learn. We generally recommend 500+ rows of input data.

[@notnikhilkanwar]

Hey! i like your video i just have one question does this generated data have the same distribution like the training data ?

[@gretel_ai]

The distribution of our synthetic data matches the real world data very closely- you can use the SQS report to compare the real world vs. synthetic data. Cheers * gretel.ai/blog/how-accurate-is-my-synthetic-data

[@mersancanonigo2933]

Thank you sir...

[@gretel_ai]

Thanks Mersan! Check out the new version of the video here: kzbin.info/www/bejne/lZ2umKuoeairqKM

[@chandrimadebnath2859]

sir, i just start my work on synthetic data geneartion. can you help me on like what are the prerequisite needed. it will be a grwat help for me. thank you.

[@gretel_ai]

Hey Chandrima- here is a link to our latest docs on pre-requisites. All you will need is an environment that can run Python, and you can use the Python command line interface (CLI) or SDKs to build synthetic data. docs.gretel.ai/environment-setup

[@hinaque4505]

I am really interested in creating synthtic data, what ML basics should I know before trying this?

[@gretel_ai]

Hey Hina! No ML experience is required to try Gretel, and you can get started building your first model through our web interface if you'd like at console.gretel.cloud. If you would like to read up on how Gretel works under the hood, here's a technical blog that's a good start: gretel.ai/blog/using-generative-differentially-private-models-to-build-privacy-enhancing-synthetic-datasets-from-real-data

[@bthapa94]

where can I access the collab code above to play around in my environment?

[@gretel_ai]

Here are links to our Colab notebooks for our SDKs and use cases. Good luck! docs.gretel.ai/synthetics/synthetic-sdk-notebooks

[@edwinfromayazona5214]

Brilliant Can we get the notebook used in this tutorial?

[@gretel_ai]

Here you go, let us know how the notebook works for you! github.com/gretelai/gretel-blueprints/blob/main/docs/notebooks/create_synthetic_data_from_a_dataframe_or_csv.ipynb

[@positive_thinker]

@@gretel_ai it works perfectly ,thanks

[@bholobadshahg11]

Hi, I followed the example. However, model.build() generates an error, KeyError: 'checkpoint_dir'

[@gretel_ai]

Hey Owais- feel free to connect with our team on Slack (gretel.ai/slackinvite) or send us an email at [e-mail]. Also, here's a link to a notebook which is tested with our latest code- github.com/gretelai/gretel-blueprints/tree/main/docs/notebooks Best of luck!

[@leandroborges7941]

When I try to do all this in Jupyter Notebook, it gives error "Failed to validate credentials"

[@gretel_ai]

Leandro- you need to generate an API key from console.gretel.cloud and copy that key into your browser. Good luck! If you have any questions, you can reach us on our Slack community at gretel.ai/slackinvite

[@efremyohannes2334]

Is there a way to produce the model performance report for Time Series synthetic data generation as with other examples shown?

[@gretel_ai]

You can generate and view the model performance report the same way as with the other examples- e.g. import IPython from smart_open import open IPython.display.HTML(data=open(model.get_artifact_link("report")).read()) Good luck!

[@randomruzan13]

Can we create synthetic data for parallel aligned sentences?. Since it will be a trial to create synthetic data for translation purpose.

[@gretel_ai]

Yes, you could create two columns (sentence1, sentence2) that you can use for training and synthesis. Good luck and feel free to email us if you have any questions hi (at) gretel.ai