Simulate throttling of the OpenAI API
Last modified on February 5, 2025
This sample contains a preset and mocks to simulate the throttling of the OpenAI API. As more and more applications use OpenAI API to use AI, it's important that developers can verify that their apps can properly handle cases when the API is throttled. Especially when the app is using lower-tier API plans, throttling is more likely to happen so validating the app's behavior is crucial to ensure great user experience. Using this preset you can simulate throttling of the OpenAI API and see how your app will handle it. The preset includes 3 different throttling scenarios: exceeded tokens per minute, exceeded number of requests per minute and service overload, which includes dynamic simulation of handling retry-after. Proxy will simulate throttling the API using one of these modes at random.
View on GitHub Download from GitHubReferences
The tutorial will introduce you to the Dev Proxy and show you how to use its features.
Instructions on how to change the mocks file used by the Dev Proxy.
Instructions on how to configure the Dev Proxy to use a different configuration file.
The OpenAI API reference documentation.