Multiple Datasources for Springboot : Neo4j 4.x & Mysql

13 min readSep 12, 2022

I recently tried (and failed for quite some time) to configure multiple datasources for a Springboot project, for several reasons :

if the databases belong to different types (a Mysql database and a Neo4j database for example), they both need their own driver and transaction manager, configured manually.
if the databases belong to the same type (2 Mysql databases for example), they also need their own entity manager factory, which adds some manual configuration to do.
if there is a good amount of manual configuration examples and documentation for Neo4j with the ‘old’ Springboot dependency org.neo4j:neo4j-ogm, I could not find any for the more recent spring-boot-starter-data-neo4j dependency.

That is why after a bit of work, I’m happy to share this little guide to help you overcome these problems and understand a bit more how it works. It is based on an example Springboot project using 4 databases (2 Mysql, 2 Neo4j), the code of which will be detailed.

Here are the versions of the different technologies used for this project :

Java : 17 (versions from Java 11 work fine)
Maven (3.8)
Spring-boot-starter-parent : 2.7.3
Mysql : 5.6 (this is an old version, but it works fine on more recent versions)
Neo4j : 4.x

What does a datasource configuration take ?

Thanks to Springboot, when using a single database for a project, almost everything is auto-configured and easy. This advantage comes with the inevitable drawback of not seeing (and as a result, often not knowing) all the little things happening under the hood. I do not know enough to detail all these processes, so I will stick to the basics :

The connection between the Springboot project and the database is configured through an appropriate driver, usually provided by the associated dependency. If the database connection is auto-configured by Spring boot, the connection informations (the database URI, a username, a password, and optional parameters) are provided to the driver through the application properties. It is nevertheless possible to manually configure the driver for more customization.
Each data operation ordered by the Springboot project is conveyed to a database through a Transaction. If the operation is successful, then the transaction is committed. Otherwise, a rollback is initiated so the data are back to their initial state.
The transaction commit and rollback is managed by a Transaction Manager object. The latter is either auto-configured by the framework, or manually configured if more customization is needed.

Multiple datasouces configuration plot-twist

Now here is the thing : using multiple datasources in a Spring project implies instantiating multiple datasources and transaction manager beans. The default configuration would give these beans the same name, and Spring would not know when to use one or the other. That is the part where a bit of manual configuration is necessary. We have two main options to overcome the problem :

solution A : manually instantiate and name the beans, and giving one of the beans the priority over the other by using the Primary annotation.
solution B : manually instantiate and name the beans, and manually configure their role (ie which objects and transactions to manage) which is my prefered option.

Why do I choose solution B over A ? because through option B, I will specifically associate entities (or packages containing entity classes) to a database and a transaction manager. It is in my opinion a cleaner way to use multiple datasources than basically telling Springboot “Use the primary datasource, or the secondary if the first attempt fails”. Furthermore, in the rare case of a project needing more than 2 datasources, option B offers more clarity to the developper about which datasource will be used in each case.

I will give an example of such a case, using 2 Mysql databases and 2 Neo4j databases as datasources for a single Springboot project.

Multiple datasources project : 1st Mysql datasource

Let’s start with the very basics : the project will first connect to only one database (Mysql), and will get more complex as we will add more datasources. Here is the application’s global architecture.

global architecture

Let’s start with the classic architecture of the project :

Nothing complicated yet : just a standard project

application.properties : connection parameters

The hostname, schema, username and password must of course be replaced by the correct values to connect to your database

Mysql1Entity - Mysql1Repository : the object - the associated empty repository

GenericService - GenericController : the basics for processing GET requests

The main class

…and with just these basic classes, a GET request to http://localhost:8080/mysql1 will get you the Mysql1 type entities from the database.

Let’s add some complexity by adding a Neo4j entity, and the necessary configuration to connect to the related Neo4j database. It is to be noted that the chapters are independent : you may also add first another Mysql database (see next chapter).

Multiple datasources project : 1st Neo4j datasource

Adding a second datasource requires numerous modifications to the code and the project architecture. Let’s start with the simplest one : adding required dependencies and connection properties.

Since both datasources will need to be manually configured, we need both Neo4j and Mysql connector dependencies.

modifying the Mysql1 connection properties and adding the neo4j properties

As you can see, multiples changes were applied to the properties :

Mysql : spring.datasource became spring.mysql1, which will help us remember which properties are related to the different datasources.
Mysql : url became jdbc-url. In order to manager multiple datasources, Springboot rely on Hikari Connection Pool, which is the tool requiring this specific modification.
Neo4j : in a similar naming fashion to Mysql, we use the custom spring.neo4j1 prefix for our properties. This implies we will have to tell Springboot how to use these non-standard properties.

Let’s talk about the new files we need to create now :

First, here is the details of the Neo4j Node and (empty) repository :

Nothing tricky here. Now, let’s dive into the datasources configuration : we basically need to give Springboot 3 indications :

which connection parameters to use for each datasource
to which repositories should each datasource be related
on which Transaction Manager should each datasource rely

In the following configuration classes, the red numbers indicate the logical order of writing (cf the explanations below the images). First, the Mysql1 configuration :

1) @Configuration : this annotations tells Springboot that this class will instantiate beans.

2) Datasource configuration based on properties :

line 18 : this method instantiate a bean, which will be named “mysql1”. Even though it is not mandatory to name the bean at this point, it will be necessary for adding the second Mysql datasource.
line 19 : the required properties for building this bean are to be found under the “spring.mysql1” prefix.
line 20 : this bean is a DataSource
line 21 : the Driver to be used is “com.mysql.cj.jdbc.Driver” (which is provided by the mysql-connector-java dependency).

3) @EnableTransactionManagement : the beans created here are allowed to manage transactions (who would have thought?)

4) Transaction Manager for Mysql1 configuration :

line 24 : this method instantiate a bean, which will be named “mysql1TransactionManager”
line 25–26 : this bean is a JpaTransactionManager, which requires the Springboot auto-generated Entity Manager Factory

5) @EnableJpaRepositories : this class will detect the Jpa Repositories in the specified package (this parameter is mandatory even if there is only one package containing repositories in your project). The specified Transaction Manager (in this example, the “mysql1TransactionManager” we just created) will be used for managing the detected Jpa Repositories related transactions.

The Neo4j1 configuration is quite similar, with just a few differences since the database type is not the same :

1) extends Neo4jDataAutoConfiguration : by extending this class, we let the auto-configuration do most of the job. We just override the methods creating the beans we need

2) @Configuration : this annotations tells Springboot that this class will instantiate beans

3) Driver configuration based on properties :

line 19–24 : fetching the properties we added to the properties file (Neo4j uri, username, and password)
line 26–27 : creating a Driver bean built with the fetched properties to configure the connection to Neo4j

4) @EnableTransactionManagement : the beans created here are allowed to manage transactions (who would have thought?)

5) Transaction Manager for Neo4j1 configuration :

line 31 : this method instantiate a bean, which will be named “neo4j1TransactionManager”
line 32–33 : this bean is a Neo4jTransactionManager, which requires the Driver configured on lines 26–27

6) @EnableNeo4jRepositories : this class will detect the Neo4j Repositories in the specified package (this parameter is mandatory even if there is only one package containing repositories in your project). The specified Transaction Manager (in this example, the “neo4j1TransactionManager” we just created) will be used for managing the found Neo4j Repositories related transactions.

The project is now almost ready to be launched. We still have 2 little modifications to apply :

add an endpoint to the controller so we can retrieve the Neo4j entities

in the service @Transactional annotations, we need to tell Springboot which transaction manager to use with each method by specifying the name of the bean we created in the configuration files.

modifications on the @Transactional annotations

When launching the application, we can see in the console that the packages have been successfully scanned and that the repositories have been found :

Multiple datasources project : 2nd Mysql datasource

Adding a second Mysql datasource is not a particularly difficult step compared to the last one, but a few points are still a bit tricky. Let’s start by listing the obvious, easy points:

we need to add an entity class, and its associated repository
we need to add to the properties file the connection information for the second mysql database
we want to add a method in the controller and the service to interact with the new datasource
we need to create a configuration class file for the new datasource

And here are the more tricky ones :

the EntityManagerFactory beans being related to only one datasource, we need to create 2 distincts EntityManagerFactory beans for each Mysql datasource : the single EntityManagerFactory created by the auto-configuration is not enough anymore.
the package-scan aiming at finding the repositories for each datasource creates a bean for each repositories found that way. That means that if the repositories for the 2 Mysql datasources are in the same package, the beans created by the 2 distinct entity manager factories will be in conflict. Thus, we need to separate the packages containing the entities and repositories for each datasource, which will make it easier to read later anyway.

Here are the project transformation we need to apply to take care of all these points :

Entities & repositories

the entities and repositories splitted into distinct packages

As you can see, I chose to split the entities and repositories into distinct packages inside each ‘mysql’ package, but it is not necessary : it just will make it easier to read if the project grows. Just to show it is not a requirement, I let the entity and the repository together in the ‘neo4j1’ package.

Properties

Nothing special here, we just added the connection properties for Mysql2 in a similar fashion we wrote the Mysql1 properties.

Controller & Service

We expose one more endpoint in the controller…

… and we create the associated method in the service. We need of course to inject the ‘Mysql2Repository’ into the service bean.

Modifications to ‘Mysql1Config’ and creation of ‘Mysql2Config’

The modifications to ‘Mysql1Config’ are all related to the EntityManagerFactory configuration (do not hesitate to click on the image to get it full size) :

line 31–38 : EntityManagerFactory creation

line 31 : we instantiate a new bean which will be named ‘entityManagerFactory1’
line 32 : this bean is a LocalContainerEntityManagerFactoryBean, and the method takes as a parameter the Datasource named ‘mysql1’ (which is already declared on lines 20–24, cf the 1st Mysql datasource configuration chapter)
line 33 : we create the new LocalContainerEntityManagerFactoryBean
line 34–36 : we configure the new entity manager factory with the ‘mysql1’ datasource, we tell it to scan the ‘mysql1’ package, and we give it a new HibernatePersistenceProvider

line 27 : we tell ‘mysql1TransactionManager’ to use the newly created entity manager factory with the annotation @Qualifier

line 17 : we configure the @EnableJpaRepository annotation with the newly created entity manager factory thanks to the ‘entityManagerFactoryRef’ parameter

And that’s it for Mysql1Config. The Mysql2config is quite a copy of this first configuration file in the same package, with any reference to Mysql1 replaced by a reference to Mysql2 :

We may now send requests to the 3 endpoints of our API and be happy with the results.

Multiple datasources project : 2nd Neo4j datasource

After the first 3 datasources configuration, I am sure you can guess what the second Neo4j datasource will require :

a new entity with its associated repository
a few connection properties
a new endpoint and the associated service method
a few modifications to the first Neo4j config file
a new config file for the second Neo4j datasource

As usual, we start with the most trivial points.

the new connection parameters :

the new entity and the associated repository :

the new endpoint in the controller :

the injection of the new repository and the new method in the service, with the name of the future transaction manager provided :

The configuration files for the 2 Neo4j datasources are the less trivial part. In a similar fashion to what we did with mysql, we need to add a few lines to the 1st config file before copying and adapting it to create the configuration file for the second Neo4j datasource.

These few lines are two bean declaring methods which we override from the parent class Neo4jDataAutoConfiguration.

The tricky part comes from the lack of documentation for using the ‘modern’ spring-boot-starter-data-neo4j dependency to configure multiple datasource including Neo4j instances : I finally got this solution and configured the two beans by reading the logs and trying to correct the errors, but the lack of documentation did not allow me to understand the exact purpose of those beans, and why they are mandatory.

Anyway, the configuration works, so here it is :

Modifications to the 1st Neo4j config file

line 43–46 : instantiation of a Neo4jClient bean with :

line43 : the bean is to be named ‘Neo4j1Client’
line 45 : the bean is based on the Driver configured in the getConfiguration1() method.

line 48–51 : instantiation of a Neo4jTemplate bean with :

line 48 : the bean is to be named ‘Neo4j1Template’
line 49 : the bean is based on the provided Neo4jClient named ‘Neo4j1Client’ (which is declared on lines 43–46)

line 22 : the detected Neo4j repositories associated with the datasource configured in this file must use the provided Neo4jTemplate named ‘Neo4j1Template’ (which is declared on lines 48–51)

Adding the second configuration file

Easy : copy the 1st configuration file, replace every ‘1’ by a ‘2’, and that’s it. More seriously, it is the reference to the connection informations, and subsequent references that must be updated :

I chose to rename the getConfiguration1() method in getConfiguration2() for more readability, but it is not mandatory.

And here it is, you may now enjoy playing with as much datasources as you wish in your Springboot project (at least as long as you play with Mysql and Neo4j).

Sources

spring data neo4j documentation from Spring
spring data neo4j documentation from Neo4j

Contact

Need for complementary informations ? Need help to concretely configure the datasources ? => write to louis.jeannejulien@gmail.com