What changes to the code needs to be made for CSV files? Does the yaml file need to have different structure as well as changes to the main code?
@cloudquicklabs15 күн бұрын
Thank you for watching my videos. Code should be modified accordingly csv read functionality of python and rest all should be same. I shall create new video on this topic soon.
@nathaniasantanigels3 ай бұрын
Is it possible for me to create a pipeline that scrapes data from Google Sheets, and whenever there's an update in the Google Sheet, the bot updates the data without overwriting the existing data? How can I ensure that when there's an update, it won't overwrite the previous data?
@cloudquicklabs3 ай бұрын
Thank you for watching my videos. It should be possible considering below points. 1. Google document event should be captured in GCP (need to figure out if it is possible) 2. We can stop overwritten with the pipeline set up.
@shreyojitdas933320 күн бұрын
sir can you please create an entire app(microservices preferably) with flask and gcp(cloud fuctions and most of the services.Please.I love your contents.
@cloudquicklabs19 күн бұрын
Thank you for watching my videos. Indeed I have plan to do more videos on GCP soon.
@Prathibha-v8cАй бұрын
I have experience as an Azure data engineer. I was recently assigned a GCP project in that project they are dealing with XML files. from GCS bucket XML file to big query table using cloud function.
@cloudquicklabsАй бұрын
Thank you for watching my videos. I have plans to create many videos on GCP ahead. I shall consider requirement as well there.
@osmanbaba148520 күн бұрын
So would the equivalent of Azure Data Factory be this? Or would it be a different google product?
@tejaspise46389 ай бұрын
Great video, where can i learn to write the script like the one used in the video(i want to learn how to use the google cloud library)
@cloudquicklabs9 ай бұрын
Thank you for watching my videos. Glad that it helped you. To get started you can check GCP documents for developers using APIs, I shall create new videos in future as well.
@theamithsingh11 ай бұрын
do an entire series that, shows how to engineer data on gcp :)
@cloudquicklabs11 ай бұрын
Thank you for watching my videos. Appreciate your valuable inputs here. I shall make this in my plan.
@ananbanerjee31112 ай бұрын
great video, thanks
@cloudquicklabs2 ай бұрын
Thank you for watching my videos. Glad that it helped you.
@andrewbateman2282 Жыл бұрын
Useful informative video. Thanks.
@cloudquicklabs Жыл бұрын
Thank you for watching my videos. Glad that it helped you.
@varra19 Жыл бұрын
@@cloudquicklabs Informative video... its previous video(where code is explained) is not having Audio. kzbin.info/www/bejne/mHmnaZSIlryNhbs. from 14 minutes
@sprinter5901 Жыл бұрын
8:27 I don't have the entry point function like you have. Its just an empty function with some comments inside.
@sprinter5901 Жыл бұрын
for those who want the code- import functions_framework # Triggered by a change in a storage bucket @functions_framework.cloud_event def hello_gcs(cloud_event): data = cloud_event.data event_id = cloud_event["id"] event_type = cloud_event["type"] bucket = data["bucket"] name = data["name"] metageneration = data["metageneration"] timeCreated = data["timeCreated"] updated = data["updated"] print(f"Event ID: {event_id}") print(f"Event type: {event_type}") print(f"Bucket: {bucket}") print(f"File: {name}") print(f"Metageneration: {metageneration}") print(f"Created: {timeCreated}") print(f"Updated: {updated}")
@cloudquicklabs Жыл бұрын
Thank you for watching my videos. Cloud is always evolutionary, it might have changed. Please find the necessary files from GitHub link shared in videos description. Note that GCP auto populating the code syntax as soon as you choose the run time at code option while creating Cloud Function.
@prashantshankavaram Жыл бұрын
Hi Anjan, thank you for the nice video. But the code given does not work. is it because the entry point has to be modified
@cloudquicklabs Жыл бұрын
Thank you for watching my videos. And thak you for sharing your inputs here.
@erinquinn685223 күн бұрын
I get an error having to do with health checks on the container: The error message you're encountering means that the Cloud Run service failed to start properly due to an issue with the container's health check. Specifically, the container did not start and listen on the required port (PORT=8080) within the allocated timeout. Anyone have solves for this?
@cloudquicklabs19 күн бұрын
Thank you for watching my videos. It must issue of proper configuration as I don't see reason for container health check here. Try from fresh once again.
@zzzmd1110 ай бұрын
Hi, Thanks for the great informative video. can you explain the flow if the data srouce is from a Rest API. Can we have a dataflow configured to extract from a Rest API to big query with dataflow without having cloud functions or Apache beam scripts involved? Thanks a lot in advance..
@cloudquicklabs10 ай бұрын
Thank you for watching my videos. Your requirement is custom ,where data source is API and you should query API to get data. And I believe in this case Cloud Function best suitable as API need invocation here.
@Prathibha-v8cАй бұрын
how to implement this same for xml data. if you have any reference please share them with me
@cloudquicklabsАй бұрын
Thank you for watching my videos. Currently I don't have this video but I would create it soon.
@CarlosMarin-lp9xe Жыл бұрын
Hi! I got this error "NameError: name 'data' is not defined". Does anyone know how to fix it? Thanks in advance.
@cloudquicklabs Жыл бұрын
Thank you for watching my videos. It looks to be code syntax issues. Please check you code again, May be you can re-use the file I shared in description
@hilo-coding-tutorials Жыл бұрын
i had the exact same issue and copy/pasted your code directly into the cloud function. What line in your code do you define this variable?@@cloudquicklabs
@iFunktion Жыл бұрын
Not sure how you managed this at all, I just get an error saying Container Failed to Start. Any tips on what might have failed because google cloud does not appear to give any help
@cloudquicklabs Жыл бұрын
Thank you for watching my videos. While I understand the difficulty to use GCP service (as community is very small), to me it looks that you have issue at setting Cloud Function, May be try creating new Cloud function once again , do follow the video carefully. All the best.
@erinquinn685221 күн бұрын
I have also gotten this error, any advice?
@varra19 Жыл бұрын
its previous video(where code is explained) is not having Audio. kzbin.info/www/bejne/mHmnaZSIlryNhbs.
@cloudquicklabs Жыл бұрын
Thank you for watching my videos. Yes there was miss in recording , Apologies. But the required code of the video can be found in video description which is missing piece in this video.