Understanding and tuning FileMaker Server Performance: PART Two
By Nick Lightbody BA(Hons) FRSA
May 29th 2015
Deskspace Systems Limited, West Sussex, United Kingdom
This document has been prepared with the assistance of Wim Decorte of Soliant Consulting Richard Carlton of RCC Alan Stirling of AST and FileMaker Inc. and I thank those folk for their valuable and in some cases time consuming assistance.
Armed with what we now understand let us next consider how to tune FileMaker Server in order to improve performance, and to do that we will need some measuring tools.
Once you can observe and measure you can experiment with the server resources and solution design in order to identify how best to improve performance.
We will first look at the following three Tools that I use:
● OS Activity Monitor (OS X) / Task Monitor (Windows)
● FileMaker Server Admin Console Statistics
● Deskspace dsBenchmark App
Then we will discuss how to use the information obtained from these tools to adjust your deployment for better performance.
The OS Activity Monitor (OS X) / Task Monitor (Windows)
The CPU history provides a visual display of the level of activity of the CPU cores, one track per core or virtual core. This enables you to observe whether the cpus are running gracefully or are overburdened.
If they are overburdened you can observe plateauing where a core is running at 100% for a period as opposed to only rising to 100% momentarily. This is slow and bad. In that case you need to make a change to prevent plateauing by either reducing load or increasing resource. Please see the illustrations in the next section.
FileMaker Server Admin Console Statistics
The FileMaker Server Admin Console offers you a range of possible statistics, from which you can select up to six at a time. There is little explanation of what these statistics can tell you so here in fig.1 is my cheat sheet of how to group the statistics for different types of use.
For our current purpose of core performance load testing we are primarily concerned with the first three statistics, which display with the colors shown on the example illustrations fig.2 & fig.3. We may look at I/O and Network performance another time.
● Remote Calls/sec (Dark Blue)
● Remote Calls in Progress (Pale Blue)
● Wait Time/call in micro seconds(Yellow)
Remote Calls/sec shows you the amount of work being undertake by FileMaker Server. So you need to be concerned when this falls as it shows that performance as in output to users is reducing.
Remote Calls in progress will generally show nothing or very little, not because nothing is happening but because the sample frequence of the statistics may not catch anything because it is happening so fast. You need to be concerned here if this value rises much above zero.
Wait Time/Call in micro seconds is the pretty much the reciprocal of Remote Calls/sec. When it rises the later falls. So observing it rise above the floor shows that performance is just starting to degrade, generally that degrading of performance will then escalate rapidly unless the load is immediately reduced and Remote Calls/sec and hence solution response will fall.
In my own FileMaker Framework I use a system of tokens which uses the average speed of completion of the most recent operations to control slowing down the requests for more operations in order to reduce load by giving server the opportunity with more time to restore normal performance.
The key thing to understand is that it can take a short time to overload server but that it will then invariably take far longer for server to subsequently work through the backlog of queued instructions and restore normal service. So any available method of taking the pressure off server when it is becoming stressed will deliver a benefit. By accepting a slightly slower service, which the user may not even notice, the user can then avoid or reduce the big delays that will arise as soon as server chokes.
The following illustrations fig.2 and fig.3 shows the FileMaker statistics I have described in both good and bad performance modes.
The following illustration fig.2 shows what plateauing and normal graceful operation look like simultaneously in the OS X Task Monitor and the FileMaker Server Statistics under OS X 10.10.
The following illustration fig.3 shows what plateauing and normal graceful operation look like when comparing FileMaker server 13 and 14 under OS X 10.9 and OS X 10.10 respectively.
Deskspace dsBenchmark App
This app was developed as a method of automatically load testing FileMaker Server in order to find the number of simultaneous users that any given deployment will support. This is used as a comparative measure, or benchmark. So if a deployment maxes out at 30 users and the same deployment normally supports 75 real users in a certain organization we can extrapolate that another server deployment which benchmarks at 60 users will likely support 150 users of the same organization.
As has been noted before, different users may have very different levels of activity and different solutions have have very different levels of efficiency hence no absolute measurement is realistic, only a comparative measurement.
It offers load testing of any combination desired of so called intense, busy and relatively inactive users.
This tool can be opened and used from any FileMaker Client or WebDirect session and so, for example, a mixture of different types of FileMaker Pro and FileMaker Go clients and WebDirect sessions can be measured simultaneously.
DsBenchmark has two modes, local and server side and hence can be used to run a single virtual client from each of a number of devices and from a number of separate WebDirect sessions and/or to instigate a number of server side virtual clients.
The activity of each virtual client is simple, it creates a pair of linked records containing random data through a relationship, using a fully featured method forming part of our normal FileMaker App Framework so this is doing much more than just making blank records, pauses and then repeats the operation until the performance slows to fall below a performant level where the average time taken to complete each operation exceeds 3 seconds.
An earlier version also randomly edited each record after creation but careful observation revealed that by far the most common load inducing activity is the creation of new records and hence this tool focuses on doing just that.
The length of the pause is determined by whether you decide to use intense, busy or fairly inactive virtual clients.
A useful way to start is to:
● Run the tool several times on your normal server when there are no real clients and check the level at which the server maxes out with “Busy” virtual users.
● Observe the chart produced by each test and when each test run has finished click the save button to save the data set and chart to your desktop.
● Remember to describe the server deployment in the “comment” field you are invited to complete. This comment forms part of every record so that different server deployments can be compared and it also provides you with a record of what the chart is recording.
● Inspect each chart and decide the user level at which the server choked. Please see illustrations fig.4 and fig.5 below.
Now having established your baseline you are ready to run the tool on another deployment in order to create a comparison between the two servers.
This can very helpful when deciding what server to upgrade to and enables you, for example, to compare the performance of cloud server services offered by different providers.
If you choose to share your data with others then it is a simple matter to combine the data sets into a FileMaker database for statistical analysis and by this method develop a realistic and evidence based method of assessing what performance to expect from any given FileMaker Server deployment.
Tuning options: the options available for you when trying to improve performance are as follows
● Adjust the FileMaker Server cache
● Increase the resources available to be used by your solution
● Improve the efficiency of your solution
● Reduce your user numbers
Adjusting the cache
● as discussed above it is a mistake to use too little or too much cache since the cache carved out of the server for use by FileMaker Server reduces that available for the operating system, FileMaker Server’s own processes, other services such as java and WebDirect sessions;
● traditionally the advice is to watch the level of cache hit and cache saved in the Admin Console statistics and then reduce the cache until the hit rate drops to slightly below 100%. The logic is that if the cache is always large enough then some of it is being wasted and hence it can be reduced;
● you can also run the dsBenchmark tool and watch the data is collects and the FileMaker Server Statistics and OS CPU activity to fairly quickly find the cache setting offering better performance and then use the traditional method to fine tune it further.
Increasing resources available, consider
● ensuring that your physical or virtual server is solely used to host your FileMaker Solution, check FileMaker Server documentation for the unnecessary and/or competing processes that should be shut down, stop colleagues using it as file server, move that email server, move that web server elswhere, free up the resources for your number one priority;
● changing rotating storage media for a Solid State Drive;
● increasing the RAM fitted to the server and reducing the number of files hosted on your server;
● perhaps a heavily used file would be better hosted on a dedicated server;
● purchasing a secure certificate to enable your WebDirect sessions to cache pages and hence run much faster.
Most performance issues arise from legacy solutions designed for a very different environment which have not been updated to modern standards. Consider the don’ts and does in Part 3, you may find some easy wins.
If you really want great performance consider a wholesale rewrite of your solution aiming to simplify its operations and reduce complexity, you will find that done the right way this can offer considerable benefits.
Such a rewrite should result in something nowhere near as complex as the original and in consequence such a process may not be as big a job as you may think. If you regard your legacy solution as a specification for what the users should receive, but not for how that should be achieved, you will be on the right track.
The modern FileMaker tool set offers the opportunity of greatly reducing complexity and correspondingly greatly increasing efficiency and hence performance.
The final option is to reduce your user numbers
In practice this may involve running say two copies of a solution on separate servers and regularly syncing them;
Ensure that inactive users are disconnected;
Bear in mind that users connecting over WAN from mobile devices instead of locally over LAN will reduce the load due to the narrower band width they can apply.
In Part Three we look at things you should do or not do when building a FileMaker solution or App to provide good performance when hosted on FileMaker Server.