KEYCLOAK Implementing Custom User Storage Provider (in-depth) | Niko Köbler (@dasniko)

[@scout23DF]

Awesome! Extremely helpful ! Thanks, Niko Köbler!

[@learnthroughcoding5554]

Awesome video! helpfull !... please provide a video for the custom ISP in Keycloak

[@enoosnajekliw]

Hello Niko, Thank you for these helpful examples. Trying out the PeanutsUserProvider tests I got an error in the test testLoginAsUserAndCheckAccessToken. The access token payload does not contain the requested attributes, if I switch to the id token however, the test succeeds. Am I missing something? Thank you in advance. Mrs. Wilke Jansoone

[@RomainHerault-q6n]

Thanks for this awesome tutorial! Is that possible to store default users in my own database but let Keycloak manage credentials (password, etc) ?

[@EdmilsonSantana-tx3ww]

Hi Niko, thanks for the explanation of your example provider. I'm just finishing to implement a redis provider. My use case only requires user to be imported to KC local storage but could easily be extended to other use cases.

[@francescobagnoli2637]

Great, thank you for these awesome keycloak tutorial! Can we have a tutorial on how to implement mandatory fields in keycloak registration or in idp-login update profile?

[@dasniko]

Have a look into the declarative User Profile SPI (still preview feature).

[@mohamedmatrouh6284]

Thank you, I really appreciate your efforts

[@TaniaKampana]

Is there any example for implementation of UserBulkUpdateProvider, ImportedUserValidation and ImportSynchronization?

[@thavathrath7022]

Great, it's helpful. many thanks 😊🙏

[@francois-xaviervits6051]

Thanks for the video. It was very helpful. Did you plan to make a video based on the new api Map Storage API that replace the deprecated User Storage SPI in the latest keycloak version ?

[@tony_0088]

Awesome video. I've seen many of your videos, and you are up there among the best regarding Keycloak. In doing a User Storage Provider similar to this here, I've noticed that every time keycloak loads a "federated user" data (for example an via Api), that same user data is loaded many times, which translates into many equal requests hitting the server containing the "federated users". In case this a known phenomenon could you please point me towards relevant documentation where I can learn more about avoiding this? Keep up the awesome work, and just so that you know, your material helped me a lot in implementing custom 2FA checks, and custom authentication flows.

[@dasniko]

See e.g. here: github.com/dasniko/keycloak-extensions-demo/blob/6b75adb50016f614f7c1c5c8b7a0c0784884e5d0/peanuts-userprovider/src/main/java/dasniko/keycloak/user/peanuts/PeanutsUserProvider.java#L46 Although I'm using currently a more advance, but also more complex version, the approach is the same by "caching" the user which is found once during one transaction and thus reduce the calls to external storage.

[@tony_0088]

@@dasniko Awesome thanks! I did think about caching the results too, but I just wanted to confirm that that is the natural behaviour and not some kind of rookie mistake I was incuring into XD. As allways, your content is AAA and extremely didactic (which is particularly helpful considering the official doc barely scratches KC possiblities). Again, I'm truly thankful, your work is amazing.

[@miguemc83]

Thank you very much for this video. I have a keycloak realm which only accepts x509 authentication. Is there any way to implement a x509 custom user provider? I need to get the userdata from a remote web service which accept only signed (with the user certificate) requests

[@שרהקצב-י4פ]

Hi Niko, thanks for your explanation. is it possible to create custom provider for groups search, in order to make the search case-insensitive?

[@CD-vo6nj]

Hi. Extremely useful example. Thanks. Do you know if it is possible to map client roles as well as realm roles. If I return them both from getRoleMappingsInternal they all get added as realm roles (even if I set isClientRole to true in the RoleModel)and getClientRoleMappings doesn't seem to get called.

[@vb1966]

Thank you very much for this video and popularization of providers for keycloak. You are my teacher :). Your Flintstone example helped me a lot to start implementing my first provider, but I had to go further with it. Namely, to implement role management, adding roles to the realm from storage, assigning, deleting mappings. The only thing that is not solved for me is the interception of the role creation event in the realm in order to instantly synchronize it with the jdbc repository. Thanks for your experience

[@toonlenkens3650]

11:47

[@ismailbouaziz3175]

Thank you for this tutorial, just I want to know which is the best embedded keycloak or standalone keycloak for this solution and how I have to make my choice.

[@dasniko]

There is no embedded Keycloak.

[@savagetadow]

Just started the learning path regarding this thread and would be very grateful for a hint on how to install the custom user federation. I know that the JAR package should be in the providers folder, but how to create this JAR file? I tried to use the default command in cmd, but it didn't work for me( Get nothing after "kc.bat build" command(

[@mingli9563]

This channel is so great!

[@sunnatillaaxakalov5784]

Thanks, good video. Do you know how to distinguish credential types ? In this video used PasswordCredentialType, how to realize PincodeCredentialType, FingerPrintCredentialType, and send request for authorization via Authzclient ?

[@sunnatillaaxakalov5784]

@dasniko, could you suggest some direction to resolve problem ?

[@timafun]

Is it possible to create some kind of User Attribute Provider attributes from which we can map to LDAP/AD users ?

[@hamza201183]

Are attributes only of type String? Or can it be any object? Imagine the scenario where the peanut user contains also objects (like a list of products, a "one-to-many" relationship to products). Can the user SPI handle this use-case? Vielen Dank!

[@dasniko]

Internally, user attributes are a MultivaluedMap, so an attribute key can have multiple values, each of them are stored internally as a string, also the user attribute type is string. If the "string" value is actually a numeric value, it can be mapped to the token(s) as a number, but this is not related to the user storage, this is token mapper related.

[@hamza201183]

@@dasnikoMany thanks for your quick answer. I will consider and study this. I'm still new to Keycloak extensions.

[@andrescorrea945]

Hi Niko, thanks for the explanation, my question now is how we can combine it with 2FA.

[@dasniko]

The same way you doing it with users from Keycloak's own database, there's no difference. The User Storage SPI ist for user data, not for authentication. If you have stored the 2FA data also in an external source, you'll have to implement the methods for validating/updating the credentials properly.

[@mikhaildorokhovich1804]

Awesome Niko!!!

[@DoisKoh]

Can you implement the User Storage Provider without handling credentials? Can I leave that to Keycloak to deal with and I just store the User's ID, username, email... ?

[@dasniko]

Yes, that's possible. Just don't implement the "CredentialInput..." interfaces.

[@iasminconstantin3825]

How can SwaggerUI can be attack to SPI implementation, or this is possible?

[@odjoktiafrapdevelopper989]

Very good tutorial video! Your different tutorials about keycloak helped me to understand how to use this tool. Regarding the user storage provider, I downloaded your jar file and imported it in keycloak by providing the link to a replication of your apis in mockoon but the connection fails. Please could you tell me what this is due to. Thanks

[@dasniko]

As I don't know what you did, I cannot tell you anything. Please understand that I cannot give support through KZbin comments. Thanks.

[@javadezaz8220]

Hello Niko Köbler. How I can use Keycloak reset password and forget password feature in my react admin panel?

[@elbozo5723]

hey niko, is there any chance you could do a video on how to customize the keycloak v2 theme?

[@izac6462]

Do you think its possible to add a button to test the connection just like the ldap privider?

[@dasniko]

generally yes, but this requires to add custom resources to the admin theme and currently the admin theme can only be extended by forking it and maintaining a whole custom admin-ui...

[@AshokSapare-y2c]

Hi Niko Kobler, can we use MongoDB as User Federation in Keycloak. Does User Storage SPI can be used for this purpose?

[@dasniko]

If you have watched the video, you should have seen that the SPI is being implemented in Java. So, if you can use Java to talk to a MongoDB (hint: yes, that‘s possible), then yes.

[@AshokSapare-y2c]

@@dasniko Thank you so much for your confirmation Niko 🙂. Somewhere I had doubt bcz document says LDAP, Active Directory and RDBMS it supports. Now it is clear.

[@mehdiben9624]

Hi, can two clients in the same realm communicate with each other on keycloak ? if yes, how ? if no, why ?

[@GlennPierce]

Hi thanks just a couple of questions. If the peanuts service returned a jwt token that is used for future requests how does that get passed back to a client by keycloak ? Also am I correct in reading that a new mapping api has replace this now ? Do you still. Recommend using this api. Thanks againr

[@dasniko]

The peanuts service does not return a token, it's just for communication with a user store. The new store, based on a new "Map Storage SPI" is still in development and in "experimental" state. For production use, you have currently no alternative to the shown approach.

[@GlennPierce]

@@dasniko Thanks I know the peanuts service doesn't return ta token I meant if it did (like mine). I guess I could set the token as an attribute of the user and that would get exposed to clients.

[@crossfirein2000]

Hi, Thanks for sharing this wonderful article. I have tested it in my local environment and it is working perfectly. Is it possible to add the additional information which is returned from the backend REST API to the generated token. And also let me know if it is possible to add custom response in case of authentication failure. Thanks...

[@dasniko]

1. yes 2. not ootb, this requires further customizations

[@crossfirein2000]

@@dasniko Thanks.. Another thing I wanted to know that in the backend user names(most of the passwords also) are in upper case but keycloak is converting them to lower case resulting in the auth failure. Please suggest what could be the workaround. Thanks

[@islembenarab5451]

but i didnt find peanuts-user-provider in my choices when I lunched the server

[@cuongnguyenminh6235]

Thank you so much

[@ivandugandzic4014]

Hello Niko, is there option for remove Default configuration options for Unlink/Remove users from Local storage?

[@dasniko]

don't know

[@audioqube6593]

Hello Niko, i already realized a UserStorageProvider that access an external legacy Oracle Database for users with keycloak 16.1.1 but now with keycloak 17.0.0 I don't know how to tell my custom provider how to configure the additional datasource. Before the datasource was configured inside wildly but now? Please help me! I don't find anything in the documentation or forums that give me advices...Thank you in advance

[@dasniko]

The documentation, examples and also some adjustments in Keycloak core therefore will be available with version 18, I guess

[@HosseinOjvar]

Thanks man, You saved my day

[@IgorSakaki]

Amazing video, very helpful!!! Do you know if possible match a client to a specific user federation?

[@dasniko]

This is not possible, because this makes no sense in context of an IdP

[@Lamoboos223]

make realm for each client .. this is a work around

[@thed9299]

Thanks for awesome tutorial and good explanation 👍 Is it possible if I use stand-alone Keyclock(on Docker) and then implement Custom User Storage Provider via spring-boot?

[@dasniko]

Providers have always to be implemented in Keycloak, not in an external ecosystem.

[@YaroslavFedorov]

thank you!

[@xpolr]

Thanks for the video, but I get a strange bug on version 19.0.1: I compile your example, but I don't see provider settings except "Console display name", "Cache policy".

[@xpolr]

This error seems to occur only on version 19.0.1, after updating all fields are visible.

[@dasniko]

Yes, that‘s a bug in the new admin-UI, hopefully it‘s fixed completely. I recommend using the old admin-ui until KC 20, due to a lot of bugs all over.

[@TaniaKampana]

Is it expected the users to have FederationLink?

[@dasniko]

Yes, of course. Otherwise Keycloak wouldn't know that theses users are originated in the respective provider.

[@TaniaKampana]

@@dasniko The expected record in keycloak database is in the federated_user? When is it expected to be inserted in the record. In the ui seems to have the federated link, whereas when i am trying to get it from the UserModel, getFederatedLink is not in place. I am trying to understand where is the issue.

[@ashishbajpayi4]

Hi, In my organization they have a requirement to have swagger functionality support for Keycloak. So for this, they want me to get the clone of it and make the changes to the source code. So, I want to ask is it required to get the clone and make the changes in the source code and build it? Will appreciate your early response Thanks.

[@xpolr]

You can simply modify your Swagger JSON by adding the necessary keycloak calls.

[@sepact7999]

Hello Sir, thank you for the knowledge you provide for free. I'm new to Keycloak and i wanted to ask, if there is any way to deploy JAR-files, which contain javascript policies, into a Keycloak instance that is running from a docker container. I know that on my local machine, i can simply create a Jar that contains a keycloak-scripts.json file and the JavaScript policies, each policy being a single .js-file and then upload the jar pretty easily through the command line. However, I don't know how i could deploy them on a containerized instance.

[@dasniko]

You have to enable the "script-uploads" feature. See docs for details.

[@paulcalinovici8808]

I would like to extend Role mapping to add attributes like validTo, can you do a video about that, or can someone help me to find resources how to do that? Thx

[@fadighattas9769]

thanks for the video, but how we can build and include the extension as jar file in keycloak docker container

[@xpolr]

Run the maven package, you will get a .jar file in build directory, which you should place in /opt/keycloak/providers (keycloak version >= 17.0.0)

[@dasniko]

Please see the official Keycloak server developer guide.

[@abdurakhmonurazboev767]

Hello Niko. Thank you for awesome video. I am currently playing around with Custom User Storage Provider. I have a Users microservice that I want to use as actual user storage provider, but I am having a difficulty to implement endpoints in that microservice for the SPI to make RestEasy requests to get required data and information. Could you please make additional video on this theme and show what will happen in the service itself that these Resteasy Requests are going to?

[@abdurakhmonurazboev767]

@@dasniko Sure, depending on how much it will be.

[@abdurakhmonurazboev767]

@@dasniko I see. I'm afraid that would be a bit expensive for me. What about hourly p2p mentoring? Do you offer such sessions?

[@lts8683]

Thank you I hope to make a video about User Storage SPI using database directly

[@dasniko]

There's an example available in the Keycloak quickstarts repository.

[@juanmax832]

@@dasniko Please, can you paste a link? I can't find where. Thank you!

[@raulperezguevara2966]

Thanks a loo, Niko, im having an issue trying to use an oracle external db as datasource on keycloak, but the oracle module is not loading, im using the lasted driver for oracle ojdb but i also tryed the previous one 11 and 8, im using oracle 19c and for keycloak version the 15.0.2, but i cant migrate it, this is the issue im having, failure description: "WFLYJCA0041: Failed to load module for driver [com.oracle]", hope someone in the comunity can help me, i know is not the correct place to ask.

[@dasniko]

Due to licencing, you have to care about the Oracle stuff yourself.

[@rubenquiroz7315]

Hello Nico, how can I customize only the user authentication, validate that the username and password in an external web service, authenticate it even if it is not registered in keycloak

[@dasniko]

That's exactly what the User Storage SPI is here for. Nothing else is doing my example.

[@rubenquiroz7315]

Thank you. So this is perfect for the use case where users need to be authenticated through an external web service without them being registered in keycloak?

[@saguntumkar]

Nice video. Can you please also add an example covering how can we add custom JPA entity provider in Keycloak Quarkus?

[@dasniko]

Look at the Keycloak quickstarts repository... there is an example of this.

[@juroltv3119]

Hello, where to deploy this? What directory should it be stored? I mean for the built jar file

[@dasniko]

Please see the official Keycloak server developer guide.

[@Lamoboos223]

/standalone/deployments

[@dasniko]

@@Lamoboos223 This is a bad answer as there is no context with it. Your answer is only valid for the legacy and now unsupported Wildfly version. That‘s why I point to the documentation, where it is mentioned depending on the version you are using.

[@Lamoboos223]

If you are using it powered by Quarkus then /providers folder should be the folder to deploy you ext with otherwise it is /standalone/deployments

[@arturf7777]

great stuff

[@satishchauhan9788]

Hi Is there way to pass additional info to custom user storage provider ? I want to acess company id when creating new user...

[@dasniko]

You can pass anything you want into custom user attributes. Depends on your code and API where you get your data from.

[@satishchauhan9788]

@@dasniko thank you for reply, yes I have used attribute option to pass additional detail for user 👍

[@juancuzmar7578]

Thank you Niko for this amazing videos. I have a request if you don't mind. Can you talk about authorization services and add some example of it? It has been a bit difficult to understand and I think with an example and a use case it would be great. Thanks again!

[@dasniko]

No, sorry.

[@juancuzmar7578]

@@dasniko why don't you know how it works?

[@dasniko]

I know how it works, but I don't do videos about each and every topic and additionally, authZ services in Keycloak are - IMHO - badly/poorly implemented.

[@juancuzmar7578]

@@dasniko ooooh i see. thats rough

[@ismailbouaziz3175]

Hi, thank you for these awesome keycloak tutoria. Can u share your class diagram.

[@dasniko]

There is no class diagram. Should there be one? For 3-5 classes? Don't think so.

[@ismailbouaziz3175]

@@dasniko i have three classes users, roles and groups in my api I don't have a CredentialData class I have just a attribut password in class user what should do to adapt this solution to my api architecture!? thank you for your reply 🙏

[@mcmdel]

Hi, Can you make a video about Key Cloak and API Six ?

[@dasniko]

no, sorry

[@raghaveshukla7373]

Hi Niko, Great Content Indeed. But can you showcase the same using Spring Boot

[@dasniko]

Why? Keycloak extensions can't be implemented using Spring.

[@TestUser-wi5cn]

Hello There. User Storage Provider has been deprecated. Do you have any work around for this?

[@dasniko]

It‘s not yet deprecated. It‘s the only one currently available.

[@Lamoboos223]

can it work on sql server database if I provided the jdbc url, username and password instead of the api?

[@sirfinsaan]

superb video. I have one question? Is there any way to update User Personal Info like firstname lastname email etc ? keycloak 17 @Niko

[@sirfinsaan]

I posted in the keycloak group as well :/ but still finding

[@dasniko]

You need to implement a proper user adapter (using some abstract classes as base or implement the pure UserModel interface)

[@sirfinsaan]

@@dasniko Thank you, I did after implementing setAttribute but problem is that setAttribute wil take only one field as a update not more than one field when I submit... I will check more solutions

[@annapetrosyan9195]

Does someone have any update on this? I need to update username and email in my custom user storage from admin console ui.