AWS DATASYNC | On-premises to AWS | Agent Setup | Transfer Data to S3

AWS DataSync
Simplifies and accelerates moving data
Discover and move your data between on-premises, AWS, and other cloud storage with end-to-end security, including data encryption and integrity validation.
Step 1
Deploy an agent
Deploy a DataSync virtual machine to access your on-premises storage system.
Step 2 - optional
Discover storage
Run a discovery job to get recommendations for AWS Cloud storage services.
Step 3
Transfer data
Create transfer tasks to move data from on-premises locations to cloud locations.
After setting up the agent in VMware .. login to agent console the very first thing you will see aws datasync activation menu …. Press 0 to get activation key

[@venkatb-h6d]

Awesome, thank you so much for making valuable stuff for different approaches.

[@JasCloudTech]

Glad it was helpful!

[@pavanjoshi1]

I can't see any activity in the datasync console but the transfer was complete. Is there any command we need to run in order to see the output of transfer in the cosole. like in 12:27

[@JasCloudTech]

Ok .. you can check datasync-agent.log file on agent but I got you .. you want on the agent console to show all the details.. but it only shows performance, connectivity or failures..

[@pavanjoshi1]

@@JasCloudTech Thank you

[@liv2rule]

Very informative. Thanks for sharing. I have question regarding datasync agent and data transfers. If I have multiple data streams to be transfered over to AWS, Can I get these data transfers to run in parallel without wait time independently. Can single datasync agent suffice for this purpose? If yes, can you please explain how this would work.

[@JasCloudTech]

@liv2rule … single agent not possible… there will be wait time … There are 2 scenarios having multiple agents and narrowing down the data transfer scope For detailed info please check the link below aws.amazon.com/es/blogs/storage/how-to-accelerate-your-data-transfers-with-aws-datasync-scale-out-architectures/ I hope it will answer all your queries… Thanks for watching 😊

[@ssureshkumar9963]

Nice video....is it continously sync mode. Or we can exit the task once it completes the transfer.

[@JasCloudTech]

@ssureshkumar you can configure task scheduling hourly weekly … only changes to data will be copied…..🙏😊

[@sumanthbonam7279]

What is bandwidth that you have in your local when you transfer 2TB of data.. I have 5TB of data and i am using direct connect from on prem n/w to aws vpc which is having low bandwidth will this approach effects other workloads which i have on aws

[@JasCloudTech]

@sumanth to control the amount of network bandwidth that AWS DataSync will use we can configure the built-in bandwidth throttle. increase or decrease this limit while your data transfer task is running. This will minimize impact on other applications.. 🙏

[@subhasishghosh22]

i am using vmware 17.5 version. After creating the agent, VMware automatically restarts. any solution ??

[@JasCloudTech]

@subhasishghosh check for resources cpu memory allocation … check vm logs ,, check for hyper v or any subsystem that might be conflicting I hope this will give you exact reason… 😊

[@kiran.karande]

Hi JasCloudTech, I wanted to check if can we upload data directly to s3 Glacier. Also whether the AWS DataSync service is also available in Hyd region ?

[@JasCloudTech]

@Kiran yes that’s possible and AWS DataSync is available In HYD region 😊

[@amitagarwal2]

Does the agent need to be manually upgraded or does AWS take care of that?

[@JasCloudTech]

@amit AWS automatically updates your agent's software, including the underlying operating system 😁

[@Olegunnarso]

great video! Can we do this same but with MySQL database, to transfer data from phpmyadmin to aws?

[@JasCloudTech]

@olegunnarso … you mean backup … aws Datasync support shared file system /object storage …

[@Olegunnarso]

here is the task I wanna solve: Right now, I have some CSV files containing oil price data from 5 years. I wanna make some database on my computer and transfer that data to AWS, because I will need it for calculations later. and I will later do some calculations with that data and real-time data of oil prices, which I would pull using API from third party apps/webpages containing oil prices. How would you approach that problem? @@JasCloudTech

[@AntaryamiSanchari]

does datasync support On-premises MongoDB as Source?

[@JasCloudTech]

@antaryamisanchari .. you can create a trigger in MongoDB to copy MongoDB data to S3 ..

[@somethinghere2]

i have problem of Automatically get the activation key from your agent,but can not activation key (agent by vmware esxi).

[@JasCloudTech]

@matchnsp please check VMware configuration network settings outbound internet connectivity… if possible you can share screenshot in order to know the issue more specifically

[@honestlyspeaking3556]

Where did you get the key, who will tell that, half info given

[@JasCloudTech]

As I mentioned I had already setup the agent .. after setting up the agent in VMware .. login to agent console the very first thing you will see aws datasync activation menu …. Press 0 to get activation key … I hope now info is complete… thanks for watching. 🙏

[@raviteja8230]

@@JasCloudTech Hi Sir, to login into the console, I have used username as "admin" and password as "password" but it was throwing Invalid login error. I even tried giving password as "admin" but it didn't work either. Can you help me?

[@JasCloudTech]

@@raviteja8230 may I know where you have deployed your agent…

[@yogeshjs]

I provision one windows rdp server as an on prem and then download the agent and install the vmware workspace after attached agent but when i play the this i got an error box like remove the hyper-v from the system before running vmware workstation.

[@yogeshjs]

When i check hyper v its already disable on my rdp kindly help me here

[@JasCloudTech]

Ok … in this case try Get-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All Its should return state disabled Removing Uninstall-WindowsFeature -Name Hyper-V -Restart If issue persists let me know …

[@saklenshikalgar4564]

hello sir, i have one small project problem i am trying to solve it but not able to do this because of some mistakes please can you help me to solve it

[@JasCloudTech]

@saklenshikalgar ok sure … you can share details on my email [e-mail]

[@saklenshikalgar4564]

I have sent email have you seen ??

[@JasCloudTech]

@@saklenshikalgar4564 yes I have gone through it .. I will send you details with steps max by tomorrow…