Handling Large Datasets in Spring Boot: A Quick How-to Guide

When running queries in a Spring Boot application, your concern is not only the logic and structure of the data. You also have to take care of the limitations of the system in terms of memory. The standard option for this is paginated queries. You have an alternative, though, useful for certain scenarios: Java Streams. In this article, you will see how to make JPA repositories return streams and how to use them.

The Problem

Using Spring Data repository classes is the usual way to perform queries in Spring, coupled with JPA. For instance, you can define a standard query like the following one, where User is an entity mapped to some database table:

List<User> users = userRepository.findAll();

JPA with Hibernate will execute the query and load every row into memory.

In case of a large number of rows, you will get high memory usage to the point that...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE