The Test Scenarios
There are four main types of scenarios, supplemented by a fifth which is the aggregation of the previous four.
- The interactive scenarios correspond to everything that happens on the user interface in the web client.
- Batch scenarios are processes with exceptionally large volume of data.
- Exchanges/communication with external applications via interfaces
- Visualization of production, consultation, and analysis reports dashboards
It is important to simulate the tasks and activities that would occur during working hours after live.
Interactive Scenarios
The interactive scenarios are all that happens on the user interface in the web client.
For these performance scenario tests that are interactive, they are usually present because these flows are critical for the company (example creation of an order: 1000 users who create six simultaneous orders).
It can also be an interactive action that will process a large number of lines (ex the payment proposal with a large number of lines), validate that the response time is always appropriate.
There may also be situations where end users customize their workspace with additional thumbnails: if the thumbnail is linked to a complex and expensive query in terms of performance, this can cause problems.
Batch Scenarios
Batch scenarios are processes with exceptionally large volumes of data. The objective during batch tests is to see their scalability, to confirm that the time and throughput remain constant and that they do not decrease significantly.
It is possible to compare the process by running it on several types of environments to see if performance increases and to what extent. Run the processes in parallel to see if they generate competitive access issues.
For batches it is important to check that it runs in the time window.
Integration Scenarios
The first important thing is that you must have the ability to evaluate the maximum load for messages, when possible, to see if it works properly.
The other aspect is that if you use real-time interfaces, it means that in your business process, at some point to continue you will wait for a response. In processes of this type, it is important to know what the response time is and if there is nothing to improve.
Performance testing at the integration level is used to determine whether we have the right approach and the most effective.
Reporting Scenarios
And similarly for reports and analytics, it is important to validate the maximum load to see if it works as expected.
Performance tests can indicate whether you have the right models and if you have a good overall design.
Scenarios of a classic day
Finally, simulating a typical operational day by simulating tasks with activities as they would occur during working hours, for example at peak time, what users will do, which interfaces would be activated, which batches, reports are executed at the same time. This is a scenario that is often forgotten.
Preparatory elements
Now that we have discussed the different scenarios, most of them need preparation. This starts with the collection of different critical scenarios by prioritizing them, listing the response time criteria and objectives but it is also all the preparation in terms of data: the simulation of data that you will generate and the tests themselves to make sure you write and follow all activities of preparations.
The other aspect is that we set targets to make sure we have an ideal target and an acceptable maximum target, otherwise you may end up testing repeatedly without a final goal. It is therefore important to define the right boundaries to focus on the results of the solution.
Make sure you know what your business goals will be and from there you will be able to identify the right performance test scenarios.
What is especially important is that you always ensure you have realistic measured targets with response time criteria and volume criteria.
One of the criteria for passing performance tests is to realize a scenario with a classic day, taking for example the activities that will take place during peak loads which corresponds to simultaneous workloads. You will need to define the distribution of the load between scenarios, the increase in load and the permanent regime and the details of the benchmarks.
Once you have your scenarios for critical business, you can then determine what the different activities of preparation will be:
- Have a dedicated environment with production data, version, modules, volumes of data and current or expected users in which you have applied the latest platform updates and critical patches, because they bring significant performance improvements.
- You will need to prepare the simulation data.
- You must define the workflow in Pre and post Go live to manage all the performance problems that could appear: who receives and sorts, who investigates, who opens the ticket if necessary. If everyone knows what to do the processing process will be smoother and its resolution will only be improved
- Finally, you need to define the tools required for testing and monitoring.
Volumes Prerequisites
Start by documenting an initial estimate of expected users and transaction volumes. And these estimates do not need to be accurate, so a rough understanding of the order of magnitude of transactions is sufficient at this stage.
From a performance perspective, a difference between 100 or 200 transactions per hour. It will not have influence, but the difference between 100,000 or 1,000,000 or 1,000,000 and 1,000,000 is significant.
So, we must determine peak hours. It is the time of day that can cause performance problems. However, this peak workload may only occur once a month or even once a year.
Thus, the typical examples are the end of month closing or recording of weekly hours on a Friday afternoon. Also provide seasonality in the load of the system: For example, at the end of the year
During the exercise to identify peak workload times, also identify the minimum load times that may offer opportunities to choose a more appropriate time to perform integration or background tasks to spread the load more equitably throughout the day.
Consider the company’s road mad as well as the design requirements for increasing the demand on the system. Transaction and user volumes play a key role in the overall success of performance testing.
Geo Location Strategy
Thus, the cloud removes many infrastructure considerations needed in other software development projects, but a key consideration that needs to be made is to choose the right datacenter where the solution will be physically located in the world, and we know that this is not just a performance consideration.
If we focus only on performance, we can say that if users are hosted in a given region, the optimal data center is probably the one that is physically closest to those users, but many dynamic 365 solutions are implemented for global organizations and must meet the needs for users located worldwide and in this situation, the decision becomes more complex and should be guided by the locations of these users and a need to provide all users with acceptable latency.
Note that this may involve some users with less-than-optimal latency, but still within acceptable limits. Finally, as a general recommendation, it is important to have a geolocation strategy in place before designing the solution so that this discussion takes place early in the project.
User Personas Prerequisites
For successful performance testing, you must have a good understanding of the different User Attributes, including user locations, user security configurations, the data associated with each user type and the expected number of concurrent users divided by Attributes.
It is important to define the daily activity profiles for each character. Customers must also have enough user licenses to be able to realistically model the behavior during the performance tests so this must be considered.
Data Composition Prerequisites
It is important to plan the composition of the data used for performance testing. The system configuration must be realistic and based on the expected configuration of the production environment.
So, use real security configurations for test users and for transactional data work with expected data volumes in production. For example, the production data is one year, two years, so we need a certain volume here.
Use the full data migration package as a starting point for this and for Finance & Operations applications, be sure to include things like opening balance inventory quantities. From there, create a realistic data set.
Focusing only on throughput without taking the time to review the data could lead you to not get production. It is therefore important to collect information on the data that are processed
It is important to test with realistic data and with realistic configuration data. For example, if a process is evaluated without serial numbers, it can work extremely well. However, if serial numbers are required in production, the same process may suffer from performance degradation.
The data used can impact the performance of some business processes. For example, a customer was processing thousands of pickup and shipping orders for which many items were used and the appropriate number of users simulating the process. However, they used only one warehouse, whereas in the real world, the same number of orders would have been distributed in several warehouses. As a result, the use of a single warehouse caused a performance problem. Once additional warehouses were added, the team was able to focus on other areas to optimize and, after additional rounds of optimization and testing, the customer was able to launch with confidence.
It is also important to consider how the data will be randomized to reduce false positives. They are more evident when the data set used is too small. For example, using the same client for each test can reduce the number of calls to the database due to caching.
Configuration Prerequisites
It is important to ensure that the environment is configured before starting any performance testing. These configurations must be part of your runbook so that before each performance test.
You can ensure that the environment is configured according to the previous run, thereby reducing the risk of performance bottlenecks caused by incorrect configurations.
In addition, if changes are made to the configurations, they can be monitored to validate their impact on performance.
Performance Test Environment
So, choose the performance test environment that is representative of the production environment where possible and ensure you that the performance test environment is in the same region as your future production environment and keep in mind that if performance problems occur in a It is incorrect to assume that performance will improve in an environment with more resources.
So, without understanding the root cause of the performance problem, it is impossible to determine whether adding additional infrastructure will help. As mentioned earlier, it is therefore essential to understand and address performance issues quickly rather than assuming that they will disappear once you are in production.
If there are integration scenarios in the performance testing plan, all related aspects must be identified and documented.
The performance test environment should always have its dependable counterpart in integration systems. The development and configuration of these tools or environment must be considered during the planning phase.
A dedicated performance testing environment is recommended for two main reasons.
First, the performance test results are only impressive if the testers are aware of the activities taking place on the system during the test. Thus, the performance test is worthless if the unexpected process suddenly imposes additional demand on the system due to the execution of the test.
And secondly, it is common for performance tests to take place around the same time as user acceptance tests. Performance tests can take some time to run and push the system to its limits and can disrupt any other test activity.
A dedicated performance testing environment ensures that performance tests can be performed in parallel with other test activities.
Responsibilities
Regarding responsibilities there is often confusion about responsibilities in case of performance problems.
Microsoft is committed to providing a scalable platform by leveraging usage profiles, telemetry data and performing appropriate sizing. If there is a performance pb at the infrastructure level or in the standard solution, they will provide a solution: hotfix for the infra, the database.
As seen previously there are several blocks that the team built which could disrupt the platform and cause performance pbs.
It must be borne in mind that the customer knows the business objectives: he must try to optimize as much as possible the business operations (ex-import produced at night rather than during the day) to optimize the efficiency of the solution, Plan and allocate resources for performance testing.
For the Partner who builds the solution, ensure that they have a clear understanding of the customer’s processes and especially those that are critical by ensuring that all non-functional requirements have been successfully recovered and that the solution being built takes this into account. The solution implemented does not disrupt the scalability of D365FO.
During performance testing, he is there to provide the expertise to educate the customer on performance testing and assist them in performing it.
The most critical point is that all test scenarios must be provided by the customer, which is important to them, so you focus on the right scenarios and prepare the right tests.
D365FOAdviceTips 
Leave a comment