fix: handle request retries and model fallback correctly - #11624
Conversation
Summary of ChangesHello @gsquared94, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the robustness and reliability of API quota error handling. By replacing string-based error detection with a structured parsing and classification system, the application can now intelligently differentiate between temporary (retryable) and persistent (terminal) quota errors. This leads to a more refined retry mechanism, appropriate model fallbacks, and clearer user feedback, ultimately improving the overall stability and user experience when encountering API rate limits. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a significant and well-designed refactoring of the API quota error handling. Moving from string-based error detection to a structured error parsing and classification system with TerminalQuotaError and RetryableQuotaError is a major improvement in robustness and maintainability. The updated retry logic, which now leverages server-suggested delays, is much more precise. The code is well-structured, and the addition of comprehensive tests for the new parsing and classification logic is excellent.
I found a minor but important issue in the new error classification logic where a 0-second retry delay suggested by the server would not be correctly handled. I've left specific comments with suggestions to address this. Overall, this is a high-quality contribution that makes the error handling much more reliable.
gsquared94
changed the title
|
Does this also fix the issue in general where it thinks it's exhausted the quota? Even simple things (with a Code Assist Standard account) such as 'build this feature ...' which may indeed take 10 to 15 minutes and several steps but WELL within 1000 requests for the day (maybe 100 requests) it immediately fails. Essentially gemini cli seems to become unusable when there's anything that involves a little extra work in my experience so far. Trying this with a newly upgraded account. |
cornmander
left a comment
cornmander
left a comment
There was a problem hiding this comment.
Please get @bdmorgan to review as well
Sounds good! |
|
/patch stable |
|
✅ Patch workflow(s) dispatched successfully! 📋 Details:
🔗 Track Progress: |
|
/patch stable |
|
✅ Patch workflow(s) dispatched successfully! 📋 Details:
🔗 Track Progress: |
|
🚀 Patch PR Created! 📋 Patch Details:
📝 Next Steps:
🔗 Track Progress: |
|
🚀 Patch Release Started! 📋 Release Details:
⏳ Status: The patch release is now running. You'll receive another update when it completes. 🔗 Track Progress: |
|
✅ Patch Release Complete! 📦 Release Details:
🎉 Status: Your patch has been successfully released and published to npm! 📝 What's Available:
🔗 Links: |
5 participants
Second attempt for #9407. This was reverted because it didn't handle different error format being returned by the GCA service when run in Dev mode vs Prod mode. Fixed in this PR, to handle both error formats. See https://github.com/google-gemini/gemini-cli/issues/11316 for more details on that.
TLDR
This pull request refactors the handling of API quota errors to be more robust and reliable. Previously, we relied on string matching to detect specific quota error messages. This change introduces a structured error parsing system that can correctly classify quota errors as either retryable (e.g., per-minute limits, small retry delays) or terminal (e.g., daily limits).
The retry logic has been updated to use this new classification, ensuring that we only trigger model fallbacks for terminal quota errors and use the server-suggested delay for retryable errors when available.
Dive Deeper
The core of this change is a new error classification system:
parseGoogleApiError: A new function in packages/core/src/errors.ts that can parse deeply nested and stringified Google API error responses, extracting the structured error details.TerminalQuotaErrorandRetryableQuotaError: New custom error classes to represent different types of quota issues.classifyGoogleError: A function that uses parseGoogleApiError to analyze 429 errors and wrap them in the appropriate custom error class.The retryWithBackoff function has been refactored to:
TerminalQuotaErrorto trigger theonPersistent429fallback mechanism.RetryableQuotaErrorand use theretryDelayMsfrom the error for a more accurate backoff period.The UI hook
useQuotaAndFallbackhas also been updated to useinstanceof TerminalQuotaErrorfor its logic, simplifying the code and removing the need for the old isProQuotaExceededError and isGenericQuotaExceededError functions, which have been deleted.Reviewer Test Plan
Using GCA auth
Clone a large repo like https://github.com/kubernetes/kubernetes
Create an OTA at http://rhea.
Run
gemini-clifrom this branch, login with OTA oauth.Start making queries like this to get it to exhaust quota:
On quota exhaustion you should see the model fallback error.
Using Gemini API key
gemini-clifrom this branch, use this Gemini API key for authretryInfoobjects) instead of failing immediately.geministable build output - retries a bunch of times, then fails with a messy error messageTesting Matrix
Fixes: