Self-service portal for employees

General Discussion
#1

I want to develop a self-service portal for employees where employees will be able to view their pay-subs, personal info etc. They will also be able to edit employee (self only) info, submit leave application etc. The number of internal users can be huge, 100,000 to 150,000 (not sure about concurrent users but it could be very closed to everyone at the peak time). This self-service portal will exchange data between itself and HR systems.

Do you have any suggestions based on your experience on what project framework should I use? what should be architecture etc.?

#2

There are many considerations to think over, I’ll try to summarize them.

The best approach is to collect the detailed requirements first, e.g.:

  • usual information engineering - map the processes and make data models using diagrams
  • not be vague about concurrent users, but rather have the management say what they consider acceptable - and then have the ability to scale if they change their minds
  • what exactly the HR system is, and how exactly will data be exchanged - will your ss portal read/write directly to the HR database, and what, can it use some REST api for that, will you send some exchange file for it to read, and when/how often will exchange take place, or is it realtime
  • do you need your own database for the ss app, what will you keep there
  • security requirements - access rights, encryption, etcetc?
  • on-premises data center, cloud - geographic distribution of the customer sites/facilities
  • who will maintain the servers, and application

Then after you analyze what has to be done, you can start to plan the architecture and technologies to be used

  • obviously, technologies need to be enterprise-grade (enterprise assurance) - well-proven languages like Java, good database, no less than Postgres and if the customer wants to feel better then they can pay MS or Oracle licenses, frameworks that have commercial support and will not be discontinued next month - this dictates the operating system choice too
  • monolith vs microservices - there is no need to complicate things just for the sake of it, but if it is a complex project or has the potential to be, and needs scalability, you are looking at docker/kubernetes orchestration, rest api, PostgreSQL, Java ecosystem (Spring Boot, maybe something else than Tomcat), Ubuntu LTS Linux distribution, possibly Kafka if there is a lot to transfer
  • alternatively, there are Microsoft, Oracle, Red Hat, SAP ecosystems, however, they carry additional license costs
    I don’t see a reason why jmix can’t be used for that as it is now, although I would love it if we would be having now what will we have at the end of this year regarding the FlowUI, so you should expect a replacement of the UI framework. But then again, applications are never done, they are upgraded as frameworks get a new version to keep them current.

Kind regards,
Mladen

1 Like
#3

Hi Mladen
Thank you for your details comments and for sharing some thoughts. Let me share with you a bit more about the project eco-system:

  • The HR system I mentioned is also my product, being developed in Jmix (FlowUI).
  • Considering to use MS SQL server as database

There will be two category of users -

  1. HR team as user that will not be much, may be about 100 to 150.
  2. Employees, between 100,000 to 150,000 users

All the functionalities are already existing in the main application (HR systems).
There are two options I see:

  • I do not create any separate portal for employee users but manage it through user roles required and assign that role to employees
  • I create a separate application to split the traffic. But In this case, any ideas on the platform would be appreciated. Note that I am more comfortable when it is Java than any other language!

Since the number of users are bit high and there are some process in HR system that will be executed by HR team users which is bit heavy (like processing summary of daily attendance data, processing salary end of month etc.), I am not sure and want thoughts from you guys who has experience in similar business scenarios that to decide whether it can be managed in a single system for both types of users or I need to separate them. The resources I am considering as a single system are as follows:

Application server
Tomcat
RAM: 128 GB, 16 core Xion processor, SSD or NVMe

Database server
MS SQL server
RAM: 128 GB, 16 core Xion processor, SSD or NVMe

Is it a risk putting this in a single system or should I split them into the core HR system and employee portal hosted in a separate server where the employee portal database is synced. Thanks for any suggestions or comments or sharing any experience.

#4

Hello,
I will put some thoughts here, in random order:

  • since you are developing the HR system, and have the source code, it makes sense to use Jmix for the ss
  • if you will use MSSQL, that means Windows server OS. While there is a version for Linux, I would not feel good with that because mssql is a “windows first” product, so the database server is Windows, but can be headless, they call it “server core” nowadays. Tomcat server should then be on Linux.
  • I think in your case, a separate system is the better solution
    – for “mass” usage like this, the interface needs to be simple, therefore simplified and different than richer interface for the HR team
    – if I’m not wrong currently for the flowui you need to change the source code to manage the access rights to various buttons etc, every time when there is a change - HR team will be asking you additional features often, so one more argument for the separation
    – ss portal has potential to grow big - managers, corporate communications and quality departments will all want to add links to procedures, policies, notifications, news … related and unrelated to the original purpose of the ss portal - therefore UI will even more deviate from the HR team UI and functions (that’s good thing if you can additionally charge for it)
    – if its separate, the hr team has its own app that will not have any load balance considerations or be slowed down
    – for the ss portal, you can create nice database user with exactly the rights it needs and then use that account to connect to the database, you will avoid a lot of security audit problems if don’t use admin (hide plain text passwords too, log everything)
  • calculate how much resources for 1, 10, 50 users you need - log some test accounts and monitor the system resource usage - that will give you a rule of thumb about the resources needed, and about needed settings for various caches etc
  • Estimating server config requirement - #3 by mortoza_khan - CUBA.Platform
  • Servers to support 1000 concurrent users - CUBA.Platform
  • how is this done with Jmix? Application Scaling - CUBA Platform. Developer’s Manual

Kind regards,
Mladen

#5

Thank you @mbucan

With regards to the database, from your experience, what would be better - uses separate database for the ss portal that will by sync with the main HR system or the store be connected to the main hr system database?

a) if a common database is used for both HR system and ss, which approach will be better - use REST or directly connect like HR system?

b) if suggested a separate database for ss portal, then what mechanism is suggested to sync data both ways?

Thanks

#6

Hello,

If we are discussing if the ss application will connect to the existing HR database, or have separate data in a separate database, possibly on a separate server then this decision needs some considerations:

  • it will be more robust and scalable if on a different database on a different server, but it adds complexity for you to implement all this, and maintain
  • a decision also depends on the architecture - if you want the ability to scale, then a way to have more databases/database servers needs to be ready (same for the application)

Which replication mechanism to use depends on the rdbms choice. Most commercial ones nowadays have replication features, but they may not be available in lower-tier editions. Step above is database clustering. There are external tools also to consider, if they work and are reliable, they can cost less than some high-tier editions. And you can always do it by yourself, especially if your case is simple - I have implemented scheduled replication few times, by using timestamps (create, delete, update) - but you don’t want to do it unless you must. However, you mentioned the word “sync” and this looks like clustering in near real time. You will have to research MS SQL about that, and best ask on their forum.

One of the benefits of REST and microservices is that you don’t necessarily need to touch a working component of a system if it’s packaged in a container, and large companies are benefiting because different teams can produce/maintain them with their own languages and tools and others don’t care as long as it works with REST. It will require more work from your side, rather than just simply connecting to the database.

You are solving a system of equations with many variables, and there is more than one solution. Whatever you choose, you need to “ensure your back” - the ability to scale, and to maintain, and that will either cost money, or your time.

Kind regards,
Mladen