Titles

How to compare planners for transport logistics?

Transport logistics comprises many processes and related tasks, which vary depending on the business area and the company's specifics. For instance, a pizza chain will primarily address issues related to last-mile delivery and the level of service, whereas a furniture factory will solve the problems of long-haul transportation and cost optimization.

Why do you need a planner?

The work of a logistician becomes more complicated as the business develops. The number of orders is growing and the product range and delivery zone are expanding; as a result, the operational workload volume is significantly increasing. It becomes more difficult to manually schedule delivery. At the same time, the company often starts to feel the necessity to analyze the transportation system, assess risks, and carry out structural changes.

A planner (scheduler), or optimizer, is a tool that enables improvements to the logistics system. This is software that allows you to automate business processes and systematically reduce transportation costs without violating the imposed formal restrictions.

Among the modern market of logistic software solutions, you can find many schedulers of various types and configurations. All you have to do is choose the most suitable one for your tasks and purposes.

The price of a mistake is often high. Choosing the wrong optimizer may lead to millions of dollars worth of losses. At the same time, a properly selected tool can provide the business with new opportunities and a foundation for long-term growth.

In order to make an informed decision, you need to discard purely marketing information and define what problems the optimizer will solve in your company. Based on your answer, you then have to compare the objective characteristics of different schedulers. But which ones? How? What do you need to pay attention to?

In this article, we are going to answer these and some other questions.

Why don't standard benchmarks work properly?

The first thing to remember when comparing software products is benchmarks.

The term "benchmark" is taken from economics and literally means "starting point." In business, it denotes a certain object that can be used as a reference or standard.

In the IT industry, a benchmark is a standardized task that allows you to determine the comparative characteristics of several objects. Benchmarks are used to test and compare devices and software.

In transport logistics, benchmarks are also used, but it primarily happens in academic environments. They use datasets presented in the public domain. You may find such data in popular sources, such as Solomon,Gehring & Homberger.

Each data set includes information about the transportation problem.

InstanceVehiclesDistanceReferenceDate
c1_10_1q10042478.95GH2001
c1_10_29042222.96CAINIAOOct-18
c1_10_39040101.36Q17-sep-15
c1_10_49039468.60Q17-apr-13
c1_10_5i10042469.18RP25-feb-05
c1_10_69943830.21Q05-sep-14
c1_10_79743372.03SCRNov-18
c1_10_89242629.91CAINIAOMar-20
c1_10_99040323.99SCRFeb-20
c1_10_109039852.44SCR10-sep-18

The Gehring & Homberger example dataset. The first column is the name of the set, followed by the best existing solution data: number of vehicles, distance, source, and date.

Organizations such as SINTEF collect information on the best existing solutions and publish them in the public domain with numerical characteristics.

Although benchmarks are constantly used in the scientific environment, they are poorly applicable to real business problems. Let's figure out why.

Academic benchmarks are tasks that are distant from real-world conditions.

There is an opinion that success in solving test tasks is direct proof of the software operations' quality. Here’s why it is incorrect to compare optimizers on academic benchmarks.

  • Benchmarks are abstract and extremely simplified compared to real business problems.

  • Benchmarks use standard and fixed datasets that have been in the public domain for many years. The optimizer can be trained to solve test tasks perfectly, but that does not guarantee that it will cope with real cases.

A transport logistics benchmark solution example
An example of a solved transport problem-benchmark
ПAn example of solving a logistics optimization problem

A benchmark solution example

Conceptual simplicity, abstractness, and standard data are characteristics that make benchmarks useful in academic work to compare different algorithms. In fact, benchmarks are a good foundation for testing and developing new ideas.

From a business point of view, this is not enough, since benchmarks do not reproduce real-life conditions in detail — and, therefore, cannot demonstrate how the scheduler will handle actual problems.

As an example, a simple optimizer performing well in benchmarks can effectively schedule the delivery of goods from a single warehouse. But as soon as the second warehouse opens, the task becomes more complex, and the tool can no longer tackle it.

At the same time, a broad profile optimizer may not be good enough at solving simple tasks, but at the same time, it can successfully cope with problems of higher complexity. In this case, testing on deliberately simplified examples will not allow you to see and evaluate the solution’s true capabilities.

What important characteristics are left out while comparing planners using benchmarks?

Routing

In practice, routes are built considering plenty of details, such as the current traffic situation. In benchmarks, solutions are already known in advance. This initial certainty does not let us evaluate how the optimizer will cope with an unpredictable situation.

Performance

For business, optimization time can be critical. Benchmarks do not take into account nor fix the time of the scheduler operation, nor do they consider memory consumption and other resources.

Scalability

In real life, business is constantly evolving. Logistics requirements are becoming more complex, and the size of problems and the volume of workload on the optimizer are constantly changing. Benchmarks do not allow you to evaluate whether the optimizer will cope with more challenging tasks and how effective it would be in the long term.

Flexibility

The optimizer must be able to solve diverse problems and easily adapt to changing requirements. Benchmarks use fixed data, which does not let you see how the scheduler adapts to various tasks.

Compliance with individual requirements

Benchmarks are not related to real business processes and do not take into account any specific requirements. For instance, they would not consider the dimensions and shapes of goods while calculating routes. It's rarely possible for the problems of a particular company to be fully reflected in standard benchmarks.

Examples of solving logistics problems using Veeroute.

The cloud-based optimizer considers a large number of details, including the number of warehouses, cargo dimensions, product type, time windows, etc.

Results of the delivery optimization by the Veeroute combinatorial optimizer

A furniture factory

  • Evening planning

  • 1 warehouse

  • Oversized cargo

  • 1-4 orders per flight

An example of solving a transportation problem using the Veeroute cloud-based optimizer

An electronics store

  • Evening planning

  • 2-3 warehouses

  • Small cargo

  • Up to 15 orders per flight

Transport logistics problem solved with the Veeroute SaaS optimizer

Food delivery

  • Planning every 2-3 hours

  • Multiple stores

  • Food transportation

  • 3-5 orders per flight

  • Narrow time windows

  • Frozen products

How to choose a planner

Let's look at what you need to pay attention to when comparing schedulers for real business problems.

  1. Real data and actionability of the result

    Use realistic parameters to objectively assess the feasibility of a solution in practice. Otherwise, it may turn out that the optimizer performs well only due to well-chosen input data.

    For instance, routes can be built with or without considering traffic jams. In the second case, the results will look more impressive, as the travel time will be significantly less. But, despite that, this will be a fictitious result, far from real-life performance.

  2. Performance and quality

    The optimizer must produce a stable quality result in the required time. Test different time limits to find out how the planning quality changes. If the scheduler can quickly get a "good" result, it's still worth making sure that it can find a "good" result with no time limits.

  3. Support

    The workflow is not immune to troubled situations. Make sure you have a reliable backup system in place to ensure system resiliency. Make sure that technical support is provided promptly and that the difficulties encountered are resolved in a timely manner without compromising your business. While testing the optimization engine in your day-ba-day operations, pay attention to how quick and effective is the support provided by the developer.

  4. Functionality

    Find out how the functionality of the compared schedulers differs. There is a fundamental difference between a universal optimizer and a highly specialized niche solution.

    Ideally, the optimizer's functionality should not only cover your current needs but should also create a foundation for future growth.

    • Explore the optimization criteria. What exactly does the scheduler optimize? Can it optimize delivery time, service level, mileage, fleet size, and driver staff? Does the set of criteria meet the needs of your business?
    • Pay attention to the details considered while planning. Does the optimizer take into account traffic jams, time windows, vehicle mileage, driver work schedules, cargo dimensions, and compatibility, and other parameters that can be significant for you?
  5. The nature of the developer's business

    Keep in mind the specifics of the developer's activity. If the optimizer is its main product, then if problems arise, you can safely count on a quick response and timely help. Avoid conflicts of interest.