Understanding and tuning FileMaker Server Performance: PART One
By Nick Lightbody BA(Hons) FRSA
May 29th 2015
Deskspace Systems Limited, West Sussex, United Kingdom
This document has been prepared with the assistance of Wim Decorte of Soliant Consulting Richard Carlton of RCC Alan Stirling of AST and FileMaker Inc. and I thank those folk for their valuable and in some cases time consuming assistance.
FileMaker Server is relied upon by many businesses and users worldwide to host FileMaker solutions which are generally written specifically for each business and which are often in mission-critical deployments.
The result of users being denied access to, or timely response from, the files hosted by FileMaker Server will be at the least inconvenient and at worst could prove to be profoundly detrimental to the business.
Hence businesses rely upon their FileMaker deployment being performant which means that it must have at least an adequate level of performance meaning here that the maximum possible number of the businesses’ users must be supported whilst providing an at least adequate speed of response at all times.
That means responding to all normal user interactions within one second, and completing more complex requests such as creating a new record or displaying a layout containing a portal or list of many records generally within three seconds and never in more than six seconds.
It is accepted that depending on the efficiency and size of the solution complex operations may take longer, the focus here is on supporting and earning the confidence of ordinary users.
We would also expect the first interactive layout of the solution to appear when a user connects by LAN within five seconds, by WAN on a mobile device within 20 seconds and by WAN using more powerful devices within 10 seconds.
FileMaker Server is a mature product efficiently delivering a wide range of services to users accessing those services through many different types of device and when it is not performant it is for one or several of three possible reasons, all of which are the responsibility of the solution owner and their advisers:
● the solution itself is too inefficient to run well and/or
● the server itself is under resourced and/or
● the server is wrongly configured.
3. Server Performance Analysis
Server performance analysis can be complex, because there can be multiple solutions on one server, or multiple files and/or tables in one hosted solution. While hosting multiple solutions/files/tables would seem like it could hurt performance, it can in fact help performance because it decreases contention between different users for the same resource. For example when locking a table’s record list so that a new record can be added. Further, it can be complex to determine even in a single solution how multiple users may be contending over a shared resource, because different layouts or scripts may use the same table in different ways.
In trying to measure and calibrate the complex it is necessary to identify and work with specific aspects which are both capable of measurement and comparison and which undoubtedly share the typical behaviour of the solution. In my experience, the single common operation at the core of the performance experienced by FileMaker Server users is new record creation and accordingly it is this aspect that we study closely and measure in Part 2 of this paper.
Many other database management systems have similar performance characteristics to FileMaker Server: performance degrades gracefully and linearly as the number of user requests increases until, at some level, performance suddenly becomes rapidly, perhaps exponentially, much worse. Generally this occurs when there is so much activity, or so many stalled requests, that one of the four main server resources is completely saturated, generally the CPU. Any extra requests received at that point just causes additional contention for the saturated resource and thus inefficiency grows because a little time is wasted as each extra request interrupts and is queued and the time to do this, when in excess of the machine’s capacity, accumulates into a significant and noticeable wait: which from a user's point of view can feel like the server becoming unresponsive.
There are four factors affecting the performance of FileMaker Server:
The design of the solution
The activity levels of the users
The server’s available resources
How the operating system allocates resources
4. The design of the solution
Modern well-designed solutions will make effective use of server resources, a poorly designed solution or perhaps one which was designed many years ago may make very poor use which will lead to performance problems. We cover some important don’ts and does in solution design in Part 3 of this paper.
5. The activity levels of the users
It is important to understand the difference between “busy” and “less active” users of a solution. There can be 100 or more “less active” users doing occasional finds, browsing and adding records on a very low powered server without any problem. But if just one of those clients does a wildcard search on an unindexed field across all the many records in a solution that could bog down the entire server for many minutes because the entire database may have to be read into RAM and scanned. Before this occurred only a small working set of recent records may have been needed, hence there is a very big difference.
A well-designed solution will not permit any normal user the opportunity to choke the hosting of the solution in this way.
Note also that WAN clients are always going to be less active than LAN clients because they suffer from more network latency, or delay in communication. So a server deployment may be able to support say 100 “less active” WAN clients but only say 30 or 50 “less active” LAN clients on exactly the same hardware.
6. The server’s available resources
The four main physical resources used by the processes that deliver the FileMaker Server solution to users are:
We also need to bear in mind that software resources are controlled by a wider range of factors such as licensing and security features.
Server security certificate considerations
The OS Activity Monitor (OS X) or Task Manager (Windows) enables you to review the CPU percentage being used by each FileMaker Server process. This is calculated as a maximum of 100% per logical core. Since a hardware core can have hyper-threading which allows one core to run two threads. So a four core CPU with hyper threading has eight logical cores and could theoretically show a process using up to 800% CPU in the activity monitor.
Memory is not the problem it once was because it is now fairly cheap but it is still important to use it correctly. The machine or virtual machine requires sufficient RAM for the operating system, FileMaker Server, Java processes called by FileMaker Server including WebDirect and FileMaker Server cache.
Also bear in mind that this is not just about the amount of memory available but also about the speed of the memory I/O and the speed of the installed RAM itself.
The FileMaker Server Admin Console permits the FileMaker Server cache to adjusted from the default setting and it is important to both provide adequate cache for server and to avoid providing more cache than server can use and thus potentially starve the operating system and other processes of sufficient RAM. The cache setting can be varied at will and is the simplest but method of fine-tuning performance. We discuss this later in Part 2
Increasing the cache may result in server supporting more users and performing better, but as you add to the cache so you deduct equivalent memory from the RAM available to the OS and FileMaker WebDirect, so a delicate balance should be struck. Logically you can expect to require more cache where the solution has large files and/or a significant number of users.
FileMaker Server 13 generally performs better with a cache setting of something less than 2 GB of RAM whereas FileMaker Server 14 appears to benefit from up to 4 GB of RAM.
Excessive amounts of cache do not appear to enhance performance and may in fact detract from performance.
Remember that FileMaker WebDirect requires an additional 32 MB of RAM for each WebDirect user and that this is taken from the memory which is available to the operating system, it does not use FileMaker Server cache.
Disk I/O speed has a crucial effect because unlike some other databases FileMaker is continously reading and writing to the file, this high level of activity results in the I/O speed of the system being potentially the limiting factor on performance.
Disk I/O speed is a combination of the speed of the I/O bus to the storage media and the speed of the storage media itself.
The storage media could be merely the slowest, a spinning hard disk, but connection via a faster I/O bus such as SATA, PCI, SCSI or FiberChannel makes a big difference.
Possible storage media include Spinning Media, SSD (Solid State Drive), or SAN (Storage Area Network.
Do remember that an Apple “Fusion Drive” a single logical drive created from a combination of cheap spinning media and a NAND SSD is not considered suitable for FileMaker Server because if either part fails the whole thing fails.
Network bandwidth is not usually a problem for LAN deployments but it can be seriously constrained on the client side in a WAN/mobile deployment and the latency on such a network can create serious performance issues, particularly if the solution has not been designed with WAN/mobile deployment in mind.
Server security certificate considerations
Under FileMaker Server 14 WebDirect can use WebSockets as an efficient fully duplex transport layer, provided the user’s browser accepts the security offered by the server when a secure https connection is used.
That security is handled by the server’s security certificate and some browsers will not accept a “self-signed” certificate, the free sort that you don’t pay for.
The effect of the browser rejecting the WebSockets transport layer is that it will not then cache the WebDirect pages it has received, hence pages have to be downloaded over WAN everytime they are called, which of course will have a significantly detrimental effect on the performance and hence the user experience of WebDirect.
If you plan to deploy FileMaker WebDirect it is sensible to ensure that you install a real secure certificate on your server and then you will be sure that your users will enjoy the full performance of which WebDirect is capable.
7. How the server operating system allocates resources
The operating system is responsible for scheduling the threads run by all the processes on the available CPU cores. This means that the different FileMaker Server processes compete with each other under the control of the OS for the CPU, RAM, I/O and bandwidth. This competition is known as contention and there are two different types.
“Solution Contention” is caused by FileMaker Server imposing necessary automatic temporary restrictions of access to specific parts of the solution whilst for example another client is undertaking an action such as creating a new record. In that case the table in which the new record is to be created is locked momentarily, and hence any other subsequent requests to create a new record in the table are queued to be dealt with in turn as the table becomes available again. This type of issue will clearly arise where a small number of heavily used resources lie at the core of a solution whereas if a solution is designed so that the impact of users actions are more widely distributed over many parts of the solution then deployment of that solution will likely be more performant.
This can be observed by use of multiple copies of the dsBenchmark tool, described below in Part 2, running simultaneously. Where a server maxes out at a certain user level it has been found that several more copies of the same tool running in parallel on the same deployment will all max out at about the same level, hence multiplying the apparent capacity of the deployment, until the level of parallel use reaches a level where it starts to hit resource contention. So the use of these multiple copies of a simple tool can be seen as representing the real deployment of a more complex multi-file solution.
If “Solution Contention” does not cause FileMaker Server to choke we must then consider “Resource Contention”. The processes run by FileMaker Server which contend for the available resources managed by the operating system are as follows:
● fmserverd: the database
● fmscwpc: custom web publishing inc FM WebDirect
● fmsased: execution of scripts on server
● fmsib: incremental backup
● fmslogtrimmer: responsible for keeping the logs trimmed to their size set in the FMS Admin Console
● fmserver_helpd: the wrapper service which is responsible for launching FMS Admin Console
These processes compete with each other and FileMaker Server has no control of how these processes are prioritised, how resources are allocated, that is entirely down to the operating system. So in this area all we can do is watch and do everything possible to ensure that the calls made by these competing processes on the operating system are always well within the available capacity of the deployment.
In Part Two we look at how you can seek to achieve this.