Skip to content

[YSQL] Parallelize postgres build make invocations #28542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[YSQL] Parallelize postgres build make invocations #28542

wants to merge 1 commit into from

Conversation

@qlsl1198
Copy link

Description

This PR implements parallelization for PostgreSQL build make invocations in build_postgres.py as requested in issue #27196.

Changes Made

  • Added ThreadPoolExecutor to parallelize work_dirs builds in make_postgres() method
  • Extracted build_work_dir() method for parallel execution
  • Used CPU core count to determine optimal worker count
  • Added comprehensive error handling and logging
  • Maintained backward compatibility with existing functionality

Performance Impact

  • Multiple work directories now build in parallel instead of sequentially
  • Build time should be significantly reduced, especially on multi-core systems
  • Optimal worker count is automatically determined based on available CPU cores

Testing

  • All existing functionality preserved
  • Error handling improved with better logging
  • Backward compatibility maintained

Fixes #27196
@qlsl1198
Parallelize postgres build make invocations
6c2c6a4 
- Add ThreadPoolExecutor to parallelize work_dirs builds in make_postgres()
- Extract build_work_dir() method for parallel execution
- Use CPU core count to determine optimal worker count
- Add comprehensive error handling and logging
- Maintain backward compatibility with existing functionality

Fixes yugabyte#27196
@CLAassistant
Copy link

CLAassistant commented Sep 10, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.
@jasonyb
Copy link
Contributor

jasonyb commented Sep 12, 2025 •
edited
Loading

As of right now, this is a duplicate of still-open #28503.

EDIT: #28503 has closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants

@qlsl1198 @CLAassistant @jasonyb