End-to-end solutions are considered to be the most effective solutions that keep pace with the changing infrastructure of every business, and the changing demands of each sector. There are many reasons for it, among which the most important are:
- Complete coverage of all business needs;
- Optimizing business’s performance and efficiency without having to look for multiple service vendors;
- Cost and time-effective, especially when dealing with complex systems or services.
End-to-end solutions can be applied in almost all industry sectors, and are suitable for every situation, especially when building a new product/service, or establishing or streamlining a complex business. One of our clients in the transport and logistics industry has come up with an idea to create a digital logistics platform that will concentrate on the driver’s performance and use big data from multiple sensors and platforms to analyze the driver’s performance and business processes. The platform itself offers a rich user interface and near real-time alerts of driver’s behavior, along with statistics.
The development of the platform included engaging several ⋮IW departments: design & development, cloud, quality assurance, testing, and on top of all of them, project management. In the following lines, you will be able to read how these departments collaborated with each other in the delivery of a solution that exceeded expectations.
Every successful project starts with careful and thorough planning. In our case, we used simple Jira workflows that do not add complexity to everyday developers’ tasks because we are fully aware that developers want to only code. However, Grooming and Planning meetings are a must for a successful project, where we estimate, and define scope, and commitment for the next Sprint.
However, only planning the tasks is not enough. The project knowledge has to be shared with all team members. In this case, we used Confluence where every member could check the detailed, structured documentation which was easy to read and follow.
The retrospective meetings were supported by Fun Retro https://funretro.io/ to share the team’s perspective on collaboration, deadlines, milestones, what is effective, what should be improved, and immediate action.
Once a solid foundation was established with rules, directions, and regular meetings set, the project continued in the second phase which was the development phase.
The development phase consisted of two parts: frontend and backend development, and it included one additional layer – the data layer. These three parts were set to run on one cloud.
Frontend development
To build the main web application with a responsive user interface we used Angular. The user interface consists of modern data visualization elements and self-descriptive elements that are easy to use. Since the platform itself has a lot of data, it should be presented in the most convenient way – tables, charts, maps, and dashboards. The libraries that we used to do data visualization and presentation include:
The main application was further supported by building 3rd party application add-ons using Vanilla JS. To meet the client’s needs and requirements, we also developed an additional front-end internal application using Vue.
Backend development
The business logic behind the solution includes the development of Web APIs using .NET Core, Lambda functions written in C#, and Windows service. In addition, we used external systems APIs to integrate the application with 3rd party systems for ELD management and trucking – leaders in the trucking industry.
As an experienced, BE team with multiple successful REST and SOAP-based services projects, we started with a clean concept, following best practices, and a 12-Factor app methodology.
The tools and libraries that were used for Web API development with .NET Core are the following: Serilog for logging, Swashbuckle for swagger documenting, and caching mechanism using Redis cache.
Data Layer
During the development phase, we faced different data requirements. The main project data is relation data that is stored in a PostgreSQL database, provided by Amazon RDS. Real-time events, that were coming from one of the external systems, were saved as NoSQL data and stored in DynamoDB which gave processing speed and scalability.
The data of one of the external systems was organized in multi-tenant tables, but there was a lack of foreign keys, which led to manually discovering relations and dependencies. The solution for this aspect included the implementation of an automated process for data synchronization between our application and the external system.
To process complex, real-time data we used Amazon Kinesis, Firehose, and Analytics with Amazon Glue for ETL processes. Besides these concepts, we also explored the Amazon Data Pipeline. You can check more info about it here https://iwconnect.com/data-management-and-etl-operations-with-aws-data-pipeline-service/
Cloud
The solution, including its development, is running on AWS Cloud. By using AWS Cloud, we were able to handle all types of requirements, increase scalability, and security in every field (development, sharing confidential content), troubleshoot, and on top of that, reduce cloud costs to a minimum. Except for essential Amazon web services, we explored and implemented several interesting services such as Cognito, Glue, etc. Here is the complete list of what we used:
- IAM feature
- CloudFormation – model infrastructure
- Cognito
- EC2
- CloudWatch
- Elastic Beanstalk
- Lambda
- SQS
- VPC
- S3
- DynamoDB
- Data Pipeline
- Glue
- Kinesis Data Firehose
- ElasticCache for Redis
- Parameter Store
On-premise
Next to Cloud, we also built an application that runs only on on-premise servers and it is meant for the client’s internal projects. Therefore, the development and QA environments were built on AWS, whereas the production was on-premise on client servers.
Quality Assurance
The planning, and development phases are further supported by end-to-end testing. Challenges might appear when there is a need to integrate more than two systems. In these cases, two-way data synchronization and validation can lead to long bug/not-bug discussions and triage. To accomplish our goal of a bug-free product, our QA engineers used Postman for API testing and Xray for test coverage and execution. These practices have decreased the time for discussion and resulted in increased team productivity.
DevOps
The development in separate layers: frontend, backend, data, and cloud requires several deployments within one day, and this may be troublesome if you are doing it manually. ⋮IW DevOps team has solved this by using continuous delivery/continuous integration processes and allowing developers to focus on their priorities. The CI/CD processes were augmented by creating specific Jenkins pipelines for all subprojects, which reduced time and increased productivity.
Main functionalities of the solution
The solution itself is a smart, modern, and easy-to-use one that saves time, and money, gives performance measures in real time, and provides valuable data for making future decisions. Some of the main functionalities include:
1. A complete, detailed overview of truck info
Users can see trucks, and loads on a map view with calculated distance, price, and other details.
2. Asset management
Asset management is an important part of day-to-day activities. Besides asset preview, the solution offers notifications and reminders for assets’ due dates.
3. Real-time communication
The system has a messaging functionality that allows real-time communication.
4. Metrics
Fleet and driver metrics are calculated and visually presented. This functionality allows easy comparison among drivers and also provides details for a specific driver in terms of rating, fuel consumption, speed over limit, idling, etc.
Besides these functionalities, the solution provides integration with other trucking and electronic logging devices. Having in mind all the data that it contains, the solution should be as secure as possible which is guaranteed on all levels.
Teamwork
Finally, the most important aspect of bringing the idea to life was teamwork. As mentioned at the beginning, the solution delivered is an end-to-end solution engaging several resources with different skills and from different departments. Regardless of the differences, the team worked as one cohesive unit, constantly consulting each other to deliver a solution that would exceed expectations. Working in such a diversified team is joyful, beneficial, and at the same time challenging.
Employing a complete end-to-end solution has many advantages, the most important is the fact that you have the same partner that you have already put trust in to provide you a solution to your challenges. ⋮IW has been delivering end-to-end solutions since 2004 and our technical consultants are devoted to delivering exponential value to our clients all the time. If you want to discuss and check how end-to-end solutions would benefit your business, reach out – we would be more than happy to discuss any topic that you have.
