Top 3 pain points for serverless developers

[@googlecloudtech]

💯 episodes and counting of Serverless Expeditions and it all thanks to you - our developer community. Thank you! 🔔 Subscribe so you never miss a serverless expedition → goo.gle/ServerlessExpeditions

[@mffalcao87s]

So Nice the dynamic and how in the final you bring the convention ideas to explain, really great

[@StarlightInsights_com]

Thanks for another great video! Your videoes are amazing for getting a fast and unintimidating understanding of GCP. 🤩 At 3:22 you mentioned that relational databases are good for aggregating data over many records, and then you mentioned MySQL, PostgreSQL, and SQL Server. Those databases are notoriously bad for doing aggregations over many records. A report here and there may be ok, but on a production database, you would avoid mixing CRUD operations and multi-row aggregations in the same database. If multi-row aggregations are essential for your app, consider a column store database instead. GCP offers both BigQuery and Snowflake out of the box.

[@TheMomander]

Thank you for the kind words, Data Maverick! You bring up a good point about data aggregation. I think it's a matter of degrees. Data aggregation can be done in multiple ways. From slowest to fastest: 1. NoSQL database. 2. SQL database. 3. Analytics database, like BigQuery or Snowflake. In the video we were talking about going from method 1 to 2. You're saying that method 3 is even faster, which I agree with. For some applications, method 2 is good enough. Others require method 3.

[@StarlightInsights_com]

@@TheMomander You are 500% right!!! Integrating an analytical database into an app takes considerable effort and upkeep. So 49 out of 50 times, it is better to "just" use the best database for the app and then take the performance hit doing aggregation.

[@Babbili]

8:47 exactly whatever on-prem they copy to the cloud, i came to a company they copy paste their on-prem k8s to gke, i'm changing it using cloud build, skaffold, cloud deploy, cloud run for some projects ...

[@TheMomander]

Wow, copy-paste deployments sound pretty risky. I'm glad to hear you're fixing that!

[@jeanchindeko5477]

Have you covered already how to implement a fully serverless data platform (or lakehouse) on GCP?

[@luillyfe]

I want to watch in an upcoming video, what Serverless option to take when going Serverless? what Serverless options is more suitable for some kind of application?

[@TheMomander]

Excellent suggestion, thank you! I'm adding that to my list.

[@davidspiess2047]

I love cloud run, but attaching a redis memory store on basic plan costs at least 50 € / month. Attaching a dedicated volume using Filestore is way to pricy with 200 € / month. Why not provide smaller and cheaper possibilities to attach volumes. Especially smaller apps would benefit a lot from cheap persistent volumes.

[@GoogleDevelopers]

Yes, there's a bit of a cost for using Cloud Memorystore for Redis. Awhile back, I produced content to help App Engine users migrate from its Memcache service to Memorystore/Redis where I give hints on minimizing costs. Search for "Migrating from App Engine Memcache to Cloud Memorystore" to find the post, video, and tutorial. As far as Cloud Filestore goes, do you absolutely need a live NAS/NFS-mounted filesystem attached to it? What is your use case? Another alternative to your architecture is to configure a vanilla Compute Engine VM running Redis with an attached persistent disk (much cheaper than Firestore and mountable on other VMs) and firewall. Yes, that's not a "serverless" solution. The team is looking into some kind of persistent volume for serverless platforms, but that's just an idea for now, hence why I asked about your use case.

[@vineethp8168]

I find it difficult to maintain a consistent redis connection with cloud run. Socket disconnects all the time. We tried heart beats as well. It's failing on cold start and also when we have minimum instances. So we can build a consistent application on cloud run. Is there a recipe on how the redis connection should be handled?

[@TheMomander]

That's interesting; I haven't heard of this problem before. Is the Redis instance in Google Cloud or elsewhere? Do you use "Memorystore for Redis" or have you installed Redis on a virtual machine yourself?

[@vineethp8168]

@@TheMomander Thanks for responding. We are using "memory store for redis" solution with Cloud run. Cloud run and Memory Store for Redis are connected to the same VPC. They connect and operate without any issues. Like I said, even heart beats are okay. But at a certain time for some strange reason the connection is lost. So all subsequent commands fail. We often joke that the issue happens when we are planning to do some demo to a client. There is no particular pattern on when it happens. Once a week or twice a week. The issue was more frequent when we had minimum instances running than when the cloud run scales to zero. I haven't found much on this issue online so was wondering if it had something to do with some configuration on VPC or redis client. We are using dart server at the moment but is planning to move to express.js in the hope that a more frequently maintained npm redis client library would give us better performance.

[@TheMomander]

@@vineethp8168 Thank you for writing this up. I agree that it would be a good idea to switch to a more commonly used runtime. Please add another comment when you've switched and let me know if it helped or not!

[@vineethp8168]

@@TheMomander we started using node JS package - redis 4.1.0 with MemoryStore for Redis. To test against what we already had, we had two instances of Memorystore running - one with the older dart Redis client and the other with the new Node J's redis client. Both of them have been running without the issue for over a month now since I messaged you the last time and then failed exactly the same time today. Which makes me wonder if it has something to do with some time bound process. How do other GCP users handle this scenario?

[@TheMomander]

@@vineethp8168 Thank you for the update. I've asked around and I have not heard anything about this. Sorry I couldn't be of more help.

[@MrArsalan1988]

sir plz explain what is dry run ?

[@TheMomander]

It can mean many things. In what context have you heard "dry run" mentioned? I don't believe we used this term it in the video.

[@bigdlamz]

Cloud Functions for Firebase need support for Python, Java & Go at minimum. Stop forcing Typescript /Javascript on the backend

[@TheMomander]

If you want to write functions in Python, Java, Go, C#, Ruby, or PHP, try out the regular Cloud Functions (without Firebase). You won't be able to use the "firebase deploy" command, but you will have more language choice.