Read millions of records from database using Java/Jdbc

In this video I explained how we can read millions of records from database table using jdbc in optimized way to improve the performance.

[@ampzamp]

gidday mate! nice work, youre the only guy to think of answering such a basic question! well done

[@vinkm89]

Thanks it's really helpful 🙂

[@sankar261]

Thanks. Nice explanation

[@saket5925]

Really helpful. I got this question for my interview 👍

[@SivaReddyJavaTechie]

Glad it was helpful!

[@NIRANJANREDDYBODDIREDDY]

Thank you so much sir.. This solution helped me a lot. Thank you ..

[@SivaReddyJavaTechie]

Thanks Niranjan. Please refer our KZbin channel to your friends and colleagues as it may be helpful for them. Happy learning.

[@gsssvr]

Good information sir Even same process will work with Postgres and spring boot?

[@rabinarayansabat9166]

Hello Sir, how to achieve it if we are using data jpa, (without getting any outofmemoryerror error)?

[@Arun-hu6ud]

What if i have a records around 4 lakh and 200 columns will this be working or is there any better way to it other than this bcoz we are going show all the data at once in the ui without help of pagination

[@jayedfarooqui3550]

How to pass a List of values in the SQL, instead of equal to(=) , I want to use "in" clause Eg. List listOfName; listOfName contains hundred of names Select City from country where cust_name in (listOfName);

[@PoojaJain-uv6wf]

Awesome

[@SivaReddyJavaTechie]

Thanks Pooja.

[@footballuniverse6522]

I have a program that reads around 1 million records from a oracle database on cloud but it takes me 2 minutes and a half... should I avoid writing too many logs? Could it slow it down, or is it the network bottlenecking me?

[@SivaReddyJavaTechie]

Mostly it could be issue with network round trip. Is it stand-alone program? What is the default result set fetch size? To identify the RCA, you can add few more loggers to understand where the time is spending.

[@johnchristopher7309]

I am having 9 crore record, what's the best method to retrieve.. this code is not visible.. can you please share

[@RudhinMenon]

thanks, but this does not work for db2 for some reason, strange

[@karthikeyanr6368]

How spring JPA will handle million of data? is it possible to fetch millions of records using findall method ? Can you please share your comments. This is interview questions. Thanks to share this kind of videos, this is really helpful to attend the interview.

[@PrinceOf34]

I'm trying to do this with JPA. I use Specification(which can combines many search criteria conditions) for search 1m records. In Repository, needed to implements JPA Specification Excutioner(?). After implemented, findAll now will has new findAll(Specification,Page requestOf(int page, int size)); It took about 0.5s for every search(empty condition), imidiately for equals search condition, 2s for many field with like %conditions%, 5s if i enable @Cacheable for keyword. And the performance go slower when you go to last page.I still think it's far from fast not enough for huge data.

[@maheshveer7131]

Hey did you get the answer??

[@Akshay-ox7dc]

Could you please create a github link for this code

[@SivaReddyJavaTechie]

Hi Akshay, Sure, I willl upload to GitHub and share you the link.

[@sampathkumarj2ee]

@@SivaReddyJavaTechie please give me GitHub details to review the source.

[@isuriprabuddhini8240]

@@SivaReddyJavaTechie hello sir, this is a very valuable video for me. Thank you very much...sir, could you please share the GitHub link?

[@gamingaayatamir3201]

Voice is too slow