Skip to content

refactor(merge,split,json): adopt streaming approach and standardize types, address gradle warnings#5803

Open
balazs-szucs wants to merge 14 commits intoStirling-Tools:mainfrom
balazs-szucs:random-misc-variable
Open

refactor(merge,split,json): adopt streaming approach and standardize types, address gradle warnings#5803
balazs-szucs wants to merge 14 commits intoStirling-Tools:mainfrom
balazs-szucs:random-misc-variable

Conversation

@balazs-szucs
Copy link
Collaborator

@balazs-szucs balazs-szucs commented Feb 25, 2026

Description of Changes

This pull request introduces several improvements and refactorings focused on memory efficiency, stream-based file handling, and code simplification, especially for handling large files such as ZIP archives and CBZ-to-PDF conversion. The most significant changes include switching from byte-array-based file processing to streaming via InputStream, optimizing CBZ-to-PDF conversion to reduce memory usage, and generalizing field access patterns for better code clarity.

File streaming and memory efficiency:

  • Added retrieveInputStream and storeInputStream methods to FileStorage, enabling streaming file reads and writes instead of buffering entire files in memory. This is crucial for handling large files efficiently.
  • Refactored TaskManager.extractZipToIndividualFiles to use streaming extraction from ZIP files, storing each extracted file directly via storeInputStream, and determining file size after storage. This eliminates the need to buffer entire files in memory.

CBZ-to-PDF conversion improvements:

  • Refactored CbzUtils.convertCbzToPdf to return a TempFile instead of a byte array, process images one at a time (reducing peak memory usage), and write the output PDF directly to disk. Also, Ghostscript optimization now writes to a new temp file if requested.

General code simplification and consistency:

  • Replaced getter method calls with direct field access in several classes (such as OAuth provider, analytics settings, and role enumeration) for clarity and consistency.

Checklist

General

Documentation

Translations (if applicable)

UI Changes (if applicable)

  • Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR)

Testing (if applicable)

  • I have tested my changes locally. Refer to the Testing Guide for more details.
@stirlingbot stirlingbot bot added Java Pull requests that update Java code Back End Issues related to back-end development API API-related issues or pull requests Test Testing-related issues or pull requests Gradle Pull requests that update Gradle code refactor Code changes that neither fix a bug nor add a feature labels Feb 25, 2026
@stirlingbot stirlingbot bot added Front End Issues or pull requests related to front-end development pr-deployed Pull request has been deployed to a test environment labels Feb 25, 2026
@stirlingbot
Copy link
Contributor

stirlingbot bot commented Feb 25, 2026

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://23.22.230.180:5803
Security Disabled

This deployment will be automatically cleaned up when the PR is closed.

@balazs-szucs balazs-szucs marked this pull request as ready for review February 25, 2026 20:09
Copilot AI review requested due to automatic review settings February 25, 2026 20:09
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines ignoring generated files. enhancement New feature or request labels Feb 25, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements significant architectural improvements focused on memory efficiency and code modernization for handling large PDF files and archives. The changes include adopting streaming I/O patterns, standardizing data types from boxed collections to primitive arrays, and updating to Jackson 3 with Spring Boot 3 compatibility.

Changes:

  • Introduced streaming file I/O via FileStorage.retrieveInputStream() and storeInputStream() methods, replacing byte-array-based operations for large files (ZIP, CBZ, PDF conversions)
  • Standardized PDF JSON model types from List<Float> and List<Integer> to float[] and int[] for better memory efficiency and reduced object overhead
  • Migrated to Jackson 3 (tools.jackson.databind.ObjectMapper) with backwards-compatible deserialization settings for primitives

Reviewed changes

Copilot reviewed 55 out of 57 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
frontend/src/core/hooks/tools/split/useSplitOperation.ts Added empty string case handling in split endpoint selection
frontend/package-lock.json Updated npm dependencies (iconify, posthog, svelte, autoprefixer, etc.)
build.gradle Refactored syncAppVersion task with inline code; reduced LINE coverage threshold from 16% to 13%
app/core/src/main/resources/application.properties Updated Spring Boot 3 error handling properties; disabled built-in ProblemDetails to use custom GlobalExceptionHandler
app/core/src/main/java/stirling/software/SPDF/model/json/*.java Changed List to float[] and List to int[] across PDF JSON models
app/core/src/main/java/stirling/software/SPDF/service/PdfJsonConversionService.java Updated to use primitive arrays; renamed isRealJobId to useLazyImages for clarity
app/core/src/main/java/stirling/software/SPDF/controller/api/converters/*.java Updated return types from byte[] to StreamingResponseBody/TempFile for large file operations
app/core/src/main/java/stirling/software/SPDF/controller/api/SplitPdf*.java Refactored split operations to use streaming ZIP output and TempFile instead of ByteArrayOutputStream
app/core/src/main/java/stirling/software/SPDF/controller/api/misc/ExtractImagesController.java Changed to streaming ZIP creation, removed intermediate ByteArrayOutputStream
app/core/src/main/java/stirling/software/SPDF/controller/api/pipeline/PipelineController.java Implemented streaming responses for single and multiple file outputs
app/core/src/main/java/stirling/software/SPDF/controller/api/pipeline/PipelineProcessor.java Added MAX_UNZIP_DEPTH=10 limit to prevent zip bomb attacks; changed character literals
app/common/src/main/java/stirling/software/common/util/CbzUtils.java Refactored to return TempFile and process images one at a time, reducing peak memory usage
app/common/src/main/java/stirling/software/common/util/PdfToCbzUtils.java Changed to return TempFile with streaming ZIP creation
app/common/src/main/java/stirling/software/common/util/PdfUtils.java Optimized single-image rendering by calculating dimensions mathematically instead of rendering
app/common/src/main/java/stirling/software/common/service/FileStorage.java Added retrieveInputStream() and storeInputStream() methods for streaming file operations
app/common/src/main/java/stirling/software/common/service/TaskManager.java Updated ZIP extraction to use streaming instead of buffering entire files in memory
app/common/src/main/java/stirling/software/common/model/ApplicationProperties.java Changed getter calls to direct field access for OAuth and analytics settings
Multiple signature/form/provider files Replaced String literals with character literals for single-character operations (e.g., '.' instead of ".")
Multiple controller/util files Pre-compiled regex patterns as static constants for reuse
Files not reviewed (1)
  • frontend/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@stirlingbot stirlingbot bot removed the enhancement New feature or request label Feb 26, 2026
@stirlingbot
Copy link
Contributor

stirlingbot bot commented Feb 26, 2026

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://23.22.230.180:5803
Security Disabled

This deployment will be automatically cleaned up when the PR is closed.

@balazs-szucs
Copy link
Collaborator Author

/deploypr

@stirlingbot
Copy link
Contributor

stirlingbot bot commented Feb 27, 2026

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://23.22.230.180:5803
Security Disabled

This deployment will be automatically cleaned up when the PR is closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API API-related issues or pull requests Back End Issues related to back-end development Front End Issues or pull requests related to front-end development Gradle Pull requests that update Gradle code Java Pull requests that update Java code pr-deployed Pull request has been deployed to a test environment refactor Code changes that neither fix a bug nor add a feature size:XXL This PR changes 1000+ lines ignoring generated files. Test Testing-related issues or pull requests

3 participants