Bug P2
Status Update
Comments
qr...@gmail.com <qr...@gmail.com> #2
FYI
qr...@gmail.com <qr...@gmail.com> #3
In my environment, the Build started succeeding on 8/30-31 and 9/2.
However, it timed out on 9/1, giving the impression that it's still unstable.
I'm receiving Cloud Build events via Pub/Sub and notifying Slack, so I can tell when a build occurs and times out.
However, it's also problematic that the logs don't appear in the Cloud Build history.
However, it timed out on 9/1, giving the impression that it's still unstable.
I'm receiving Cloud Build events via Pub/Sub and notifying Slack, so I can tell when a build occurs and times out.
However, it's also problematic that the logs don't appear in the Cloud Build history.
Description
Please provide as much information as possible. At least, this should include a description of your issue and steps to reproduce the problem. If possible please provide a summary of what steps or workarounds you have already tried, and any docs or articles you found (un)helpful.
Problem you have encountered:
I have been experiencing significant delays and timeouts during the npm install or npm ci steps in Google Cloud Build. This issue started recently, despite no changes in dependencies or build configuration, and is affecting branches that previously built successfully in under 6 minutes. The builds now hang for up to 60 minutes before timing out.
What you expected to happen:
I expected the Google Cloud Build process to complete successfully within the usual timeframe of under 6 minutes, specifically during the npm ci step for installing dependencies. This build has historically completed without issues, with consistent performance and no unexpected delays or timeouts. The build process should proceed smoothly, without hanging or failing due to extended delays during the dependency installation phase.
Steps to reproduce:
1. Create a Cloud Build Configuration:
Set up a Google Cloud Build pipeline that triggers on commits to a specific branch in a GitHub repository.
Ensure the build configuration uses npm ci to install dependencies in a Node.js environment.
2. Trigger the Build:
Commit any changes to the repository or manually trigger the build pipeline.
3. Monitor the Build Process:
Observe the build logs in the Google Cloud Console.
The build hangs during the npm ci step, which would normally complete in a few minutes.
4. Verify with Different Branches:
Attempt to build other branches that were previously successful. These branches should exhibit the same behavior,
indicating the issue is not related to specific code changes.
5. Attempt Workarounds:
Change the npm registry, increase timeout settings, switch to yarn, and test in different regions.
All these steps still result in the build timing out during the dependency installation process.
Other information (workarounds you have tried, documentation consulted, etc):
1. Switched NPM Registry: Attempted to use a different npm registry (e.g., Taobao) but encountered certificate expiration issues.
2. Adjusted NPM Configuration:
• Increased the fetch-retry timeout and retry settings.
• Forced the use of IPv4.
3. Switched to Yarn: Replaced npm with yarn for dependency installation, but the issue persisted.
4. Checked NPM Registry Status: Confirmed that the npm registry status was operational with no reported issues.
5. Attempted Build in Different Regions: Moved the build process to different Google Cloud regions to rule out regional issues.
6. Increased Logging Verbosity: Enabled detailed logging to capture more information about where the build is hanging.
7. Cleared NPM Cache: Attempted to clear npm cache and rebuild, but the issue persisted.
Seems also like that there are a plenty of people coming up with this issue:
As time goes by it seems more and more developers are getting afftected with this issue, which could bring some concerning up, but i dont see any actions on the side of Google Team.
Best regards,
M. Zaric @ openHandwerk GmbH