Hey there! So, it seems like you’re either dealing with a slow MSP API query that’s putting a damper on things, or you’re just being cautious and trying to avoid such a situation.
No worries, I’ve got you covered. In this tutorial, I’ll show you how to effectively use pagination when dealing with large data sets using the MSP REST API.
Before diving in, review the requirements outlined on the “Getting Started” page.
All good? Great, let’s get started with this “big data” party!
First, let’s decide what you want to achieve: pagination, returning 100+ records, or both.
Pagination: To paginate records when querying the MSP REST API, provide a numeric value for the startAt property or argument.
Returning 100+ Records: To retrieve more than 100 records when querying the MSP REST API, pass a numeric value to the limit property or argument. By default, the MSP REST API limits the number of records returned to 100.
Pagination + Returning 100+ Records: If you want to get really fancy with data sets containing 100 or more records, you can combine both the limit and startAt properties or arguments when making an MSP REST API request.
Let’s put this knowledge to the test and watch it deliver lightning-fast results!
We’ll use the coding example from the MSP REST API Collections tutorial using PHP to demonstrate how to implement the limit and startAt properties or arguments when querying large data sets.
Here’s the code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
<?php /*********************************** **** **** query_volunteers.php **** **** This file allows for authentication using your MSP API Key. **** ***********************************/ // Define MSP API variables $mspApiKey = 'Enter key area'; // Your MSP API Key $apiBaseUrl = "https://api.ministryschedulerpro.com/2/"; // MSP API base URL $endPoint = 'volunteers/list'; // MSP API Endpoint // If the endpoint is not defined and still equals the default of `Enter API Endpoint`, then create an empty $endpoint variable. if ($endPoint == "Enter API Endpoint") { $endPoint = ''; } // Define the 'fields' and 'where' arguments as a JSON object $data = array( "fields" => ["firstName", "lastName", "customFieldValues"], "where" => array( "customFieldValues.a5ddbcec605c7090000" => array( "comparator" => "isLessThan", "value" => "2020-01-01" ) ), "limit" => 100, // Set the 'limit' argument to 100 "startAt" => 100 // Set the 'startAt' argument to 100 ); // MSP API uses JSON objects as a means of exchange for all endpoints; therefore, json_encode the PHP $data variable value. $postData = json_encode($data); $url = $apiBaseUrl . $endPoint; // Set your MSP API Key. Also uncomment the content type if you are attempting to instantiate a request using an MSP API endpoint. $header = array( "Authorization: Bearer $mspApiKey", "Content-Type: application/json" ); // Time to initiate cURL request using MSP API variables $ch = curl_init(); $timeout = 60; // Set the URL and other options for cURL curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST'); // Values: GET, POST, PUT, DELETE, PATCH, UPDATE curl_setopt($ch, CURLOPT_POSTFIELDS, $postData); // Uncomment if you are using MSP API endpoint query parameters -- see lines 24-30 // curl_setopt($ch, CURLOPT_POST, true); // Uncomment if the previous line is uncommented. ;) curl_setopt($ch, CURLOPT_HTTPHEADER, $header); // Execute the call and return response data $result = curl_exec($ch); // Close the cURL connection curl_close($ch); // Decode the JSON response $rez = json_decode($result, true); var_dump($rez); ?> |
Save the file query-large-data.php in your preferred text editor. Make sure to replace 'Enter key area' with your actual MSP REST API Key to authenticate successfully when making MSP REST API requests.
Upload the file to your web directory and execute the PHP file by entering its URL in a web browser’s address bar.
Now you should be able to query large data sets using the limit and startAt properties or arguments with the MSP REST API.
Additionally, you can enhance the limit and startAt properties or arguments by using a do/while loop, a for/for-each loop, or other advanced methods the PHP library provides. These methods can help you efficiently handle pagination and iterate over large data sets.
However, we won’t cover those techniques in this tutorial as we focus on simplicity. But feel free to explore and experiment if you’re up for the challenge!
You can apply these steps to other MSP collections by changing the endpoint and adjusting the query parameters.
Remember that this tutorial prioritizes simplicity, so error checking has been omitted. However, we’ll cover error handling in more advanced tutorials.
If you have any questions or encounter technical roadblocks along the way, don’t hesitate to ask for help.
Let the good times roll with large data sets!
P.S. If you’re not a fan of PHP, don’t worry. You can check out the “Paginating and Querying Large Data Sets Using Python.“ tutorial for more options. 😉