feat(dart): Add Fury Dart Support #2112
Conversation
…not in LITTLE_ENDIAN mode.
…taStringEncoder::encodeGeneric
…taStringEncoder::encodeGeneric
… names and avoid redundant computation
… names and avoid redundant computation: add import
dart/README.md
Outdated
|
|
||
| ```dart | ||
| Fury fury = Fury( | ||
| xlangMode: true, |
There was a problem hiding this comment.
Support native serialization and xlang serialization both can be very complicated. Since the type system of xlang and native can be very different. Will xlang mode cover 95% serialization scenerios in dart? If so, we can just drop native serialization support
There was a problem hiding this comment.
I'm also OK with both native serialization and xlang serialization, it makes the implementation more versatile. Just a little concern about the work when we need to update the protocol in the future.
There was a problem hiding this comment.
Indeed, considering Dart's type system, I believe XLANG mode can cover most serialization scenarios. The Dart official library provides a considerably more streamlined set of types compared to Java. In fact, to support certain XLANG types, we've had to implement some custom types in our Fury Dart implementation.
There are a few types worth discussing:
Types currently in Fury but without XLANG support:
- DateTime (Dart's primary temporal type, equivalent to Java's LocalDateTime)
- Duration (not yet supported in XLANG)
- Decimal (not provided by Dart core, but available via mature community packages like decimal)
Types not mentioned in Fury's type system:
- BigInt (from dart:core)
- Dart Record (similar to Java 14's record type) - I should note that I've encountered challenges using Fury with Java 14+ versions, seemingly due to
Unsafesupport issues.(If I got it wrong at this point, please let me know) - Uri (from dart:core)
I believe it's feasible to use the XLANG approach for types that currently lack XLANG support without significant issues. Looking at the overall picture, XLANG mode already provides high coverage of data types needed for most Dart serialization scenarios.
Given this analysis, it seems reasonable to focus our efforts on XLANG mode rather than implementing a separate native serialization approach.
There was a problem hiding this comment.
Hi @LouisLou2 , thank you for the comprehensive response and insights. If we decide to focus on dart XLANG mode, we could remove xlang mode related code and document in this pr
|
@LouisLou2 Awesome Work! Thanks for contributing to Apache Fury with this amazing Dart support. |
dart/README.md
Outdated
|
|
||
| The implementation is organized into three main components: | ||
|
|
||
| 1. **SourceGen**: Located at `dart-fury/packages/fury-core/lib/src/code_gen` |
There was a problem hiding this comment.
codegen is a common abbr in computer science, we could use it directly without snake case
There was a problem hiding this comment.
Thanks for pointing that out. Additionally, there might be mixed usage of "source_gen" and "codegen" in the project and README. I will update these as well and then commit the changes.
dart/README.md
Outdated
| 1. **SourceGen**: Located at `dart-fury/packages/fury-core/lib/src/code_gen` | ||
| Handles static code generation for serialization/deserialization. | ||
|
|
||
| 2. **FuryCore**: Located at `dart-fury/packages/fury-core/lib/src` |
There was a problem hiding this comment.
fury-core is the naming style in java, is this an idiom in dart?
There was a problem hiding this comment.
I'm not familiar about dart, will the strcuture like lib/fury/core be better?
There was a problem hiding this comment.
Are you suggesting to change the directory structure to dart-fury/lib/fury/core and dart-fury/test? I actually considered this approach, which would eliminate the need for Dart workspace features and simplify the structure.
However, my initial intention was to leverage workspaces to completely isolate the testing environment and simulate real-world usage scenarios, especially since the package involves complex codegen functionality. That's why I decided to make the test component a separate workspace, placing it at the same level as the core component under the packages directory.
I think we can maintain this structure but change the hyphenated names to snake_case. After reviewing Pub workspaces (monorepo support)
it seems that even independent packages should follow standard directory naming conventions using snake_case. So I believe we should make this change - the outer dart-fury directory should also be renamed to dart_fury or fury_dart.
There was a problem hiding this comment.
I mean we could rename fury-core to fury, and keep fury-test as it is. This is the flutter project structure:
| @@ -0,0 +1,6 @@ | |||
| // import '../vault/fury_core.dart'; | |||
| // | |||
There was a problem hiding this comment.
example could be moved to dart/example
|
dart/test/dart_fury_test.dart is empty, do we still need this? Seems all tests should go to |
| import 'package:meta/meta_meta.dart'; | ||
|
|
||
| @Target({TargetKind.constructor}) | ||
| class FuryCons { |
There was a problem hiding this comment.
Cons is not common as an abbr, we'd beeter avoid such name
| import 'package:fury_core/src/memory/byte_reader.dart'; | ||
| import 'ser/ser.dart' show Ser; | ||
|
|
||
| abstract class BaseFury{ |
There was a problem hiding this comment.
| abstract class BaseFury{ | |
| abstract class BaseFury { |
Do we need a space between class name and {?
| static final Map<int, ConstructorParams> _clsIdToUnnamedCons = HashMap(); | ||
| static final Map<int, FieldsCacheUnit> _clsIdToFields = HashMap(); | ||
|
|
||
| // _typeToTypeSpecUnnullable 和_typeToTypeSpec1Unnullable 这里只是记录分析过的类型，但是nullable信息不在其中， |
There was a problem hiding this comment.
Could we use English comments, so that more contributors can collaborate on this?
| import 'package:meta/meta_meta.dart'; | ||
|
|
||
| @Target({TargetKind.field}) | ||
| class FuryKey { |
There was a problem hiding this comment.
Could we add docs/comments for this class? Without commens, it's not easy to understand what is it used for
...kages/fury-core/lib/src/code_gen/analyse/impl/annotation/fury_class_annotation_analyzer.dart
Outdated
Show resolved
Hide resolved
| import '../../../excep/fury_gen_excep.dart'; | ||
| import '../../annotation/location_level_ensure.dart'; | ||
|
|
||
| class FuryClassAnnotationAnalyzer { |
There was a problem hiding this comment.
If one of the parent dir is fury, the classes can be naed without Fury prefix.
And maybe fury_class_annotation_analyzer.dart could be renamed to class_annotation_analyzer.dart
| // } | ||
| // if (unnamedCons == null){ | ||
| // throw FuryGenExcep.noUnamedConstructor(locationMark.libPath, locationMark.clsName); | ||
| // } |
There was a problem hiding this comment.
Maybe use english comments
| import 'package:fury/src/datatype/int32.dart'; | ||
| import 'package:fury/src/datatype/float32.dart'; | ||
|
|
||
| class SerializerDirector { |
There was a problem hiding this comment.
This name is oncommon, we may need a new name
dart/README.md
Outdated
| ```dart | ||
| import 'package:fury/fury.dart'; | ||
| part 'some_class.g.dart'; |
There was a problem hiding this comment.
we could use example as package name. some_class gives an impression that it must be same with SomeClass
…ere already made in a previous commit 2. Merge related exception definitions in fury/lib/src/codegen/exception into a single file to avoid creating too many files 3. Rename some_class.dart and some_class.g.dart in README.md to example.dart and example.g.dart 4. Rename SerializeDirector and DeserializeDirector to SerializeCoordinator and DeserializeCoordinator respectively 5. Update .gitignore to avoid committing generated files in fury-test, but keep .g.dart files in dart/example and dart/fury/example
| static const namedStruct = ObjTypeRes(ObjType.NAMED_STRUCT, false); | ||
| static const unknownStruct = ObjTypeRes(ObjType.UNKNOWN_YET, false); | ||
|
|
||
| final ObjType objType; // null 表示明确不支持 |
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive Dart language support for Apache Fury to enable efficient serialization for Flutter applications. Key changes include adding build configuration files for code generation, new analysis and linting options, and updated documentation with usage examples and changelogs.
Reviewed Changes
Copilot reviewed 239 out of 251 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| dart/packages/fury-test/build.yaml | Configures the builder for generating test output files |
| dart/packages/fury-test/analysis_options.yaml | Adds recommended linting rules and analyzer exclusions for tests |
| dart/packages/fury-test/README.md | Provides updated instructions on running tests and the testing approach |
| dart/packages/fury-test/CHANGELOG.md | New changelog file for documenting the initial version in tests |
| dart/build.yaml | Configures the builder for generating production code files |
| dart/analysis_options.yaml | Sets analyzer rules for the main project |
| dart/README.md | Updated documentation covering an overview, usage examples, and project details |
| dart/CHANGELOG.md | New changelog file for documenting the initial version of the project |
Files not reviewed (12)
- dart/.gitignore: Language not supported
- dart/example/enum_example.dart: Language not supported
- dart/example/enum_example.g.dart: Language not supported
- dart/example/example.dart: Language not supported
- dart/example/example.g.dart: Language not supported
- dart/example/nested_collection_example.dart: Language not supported
- dart/example/nested_collection_example.g.dart: Language not supported
- dart/example/typed_data_array_example.dart: Language not supported
- dart/example/typed_data_array_example.g.dart: Language not supported
- dart/note/analysis/constructor.txt: Language not supported
- dart/note/analysis/general.txt: Language not supported
- dart/packages/fury-test/.gitignore: Language not supported
Comments suppressed due to low confidence (1)
dart/README.md:32
- The import statement here is inconsistent with other usage examples that reference 'package:fury_core/fury_core.dart'. Consider aligning the import path to avoid confusion.
import 'package:fury/fury.dart';
There was a problem hiding this comment.
Pull Request Overview
This pull request adds Fury Dart support for Flutter applications by implementing a comprehensive serialization and deserialization mechanism along with static code generation support. Key changes include the addition of build configuration files for code generation, new analysis options and documentation updates for testing and usage.
Reviewed Changes
Copilot reviewed 239 out of 251 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| dart/packages/fury-test/build.yaml | Introduces a build config for generating files from Dart source entities. |
| dart/packages/fury-test/analysis_options.yaml | Sets up analysis options and lint rules for the Fury Test package. |
| dart/packages/fury-test/README.md | Provides test suite descriptions and instructions for running tests. |
| dart/packages/fury-test/CHANGELOG.md | Adds initial version changelog for Fury Test package. |
| dart/build.yaml | Adds build configuration for the main Fury package similar to that used in Fury Test. |
| dart/analysis_options.yaml | Configures relative import error handling for the main package. |
| dart/README.md | Documents overall project usage examples, details of the implementation approach, etc. |
| dart/CHANGELOG.md | Adds an initial changelog for the Fury Dart implementation. |
Files not reviewed (12)
- dart/.gitignore: Language not supported
- dart/example/enum_example.dart: Language not supported
- dart/example/enum_example.g.dart: Language not supported
- dart/example/example.dart: Language not supported
- dart/example/example.g.dart: Language not supported
- dart/example/nested_collection_example.dart: Language not supported
- dart/example/nested_collection_example.g.dart: Language not supported
- dart/example/typed_data_array_example.dart: Language not supported
- dart/example/typed_data_array_example.g.dart: Language not supported
- dart/note/analysis/constructor.txt: Language not supported
- dart/note/analysis/general.txt: Language not supported
- dart/packages/fury-test/.gitignore: Language not supported
Comments suppressed due to low confidence (1)
dart/README.md:32
- [nitpick] The import statement in this usage example differs from the one in the PR metadata (which uses 'package:fury_core/fury_core.dart'). Consider using a consistent package name across examples to avoid confusion.
import 'package:fury/fury.dart';
| @@ -0,0 +1,12 @@ | |||
| import 'package:fury/src/const/obj_type.dart'; | |||
|
|
|||
| class ObjTypeRes{ | |||
There was a problem hiding this comment.
It's not clear about what Res means. Could we avoid abbr?
| import 'package:meta/meta_meta.dart'; | ||
|
|
||
| @Target({TargetKind.function, TargetKind.method}) | ||
| class DoNotModifyReturn { |
There was a problem hiding this comment.
What does this mean? Could you add a comment?
| @@ -0,0 +1,25 @@ | |||
| import 'package:fury/src/exception/fury_exception.dart'; | |||
There was a problem hiding this comment.
How about merge related exception together like https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/build_system/exceptions.dart
| // --j; | ||
| // } while(compare(j, pivot) > 0); | ||
| // if (i < j) { | ||
| // // 交换元素 |
There was a problem hiding this comment.
Use english comments, or remove commented code
| import 'package:fury/src/resolver/impl/ref/ser_map_resolver.dart'; | ||
| import 'package:fury/src/resolver/impl/ref/ser_no_ref_resolver.dart'; | ||
|
|
||
| typedef SerRefRes = ({RefFlag refFlag, int? refId}); |
There was a problem hiding this comment.
Res can be recognized as Resource, we shoudl avoid such abbr in whole codebase. How about SerializationRefMeta?
I suggest to avoid all uncommon abbr in this PR. We could take https://github.com/google/json_serializable.dart/blob/master/json_serializable/lib/src/json_serializable_generator.dart as an example.
Fury favors clarity over brevity in naming conventions.
1. Modify the code comments in `fury/lib/src/codegen/analyze/entity/obj_type_res.dart` to English. 2. Remove the commented-out code in `fury/lib/src/util/extension/list_extension.dart` as it will not be used in the future. 3. Delete files in `fury/lib/src/codegen/analyze/entity` and move their contents to `fury/lib/src/codegen/analyze/analysis_wrappers.dart`. 4. Rename `ObjTypeRes` to `ObjTypeWrapper`. 5. Delete `fury/lib/src/dev_annotation/donot_modify.dart` as it is not being used. 6. Merge `fury/lib/src/exception/unsupported_type_exception.dart` and `fury/lib/src/exception/fury_mismatch_exception.dart` into `fury/lib/src/exception/deserialization_exception.dart` as they are only related to deserialization. 7. In `fury/lib/src/resolver/ref/ser_ref_resolver.dart`, rename `SerRefRes` to `SerializationRefMeta`, `SerRefResolver` to `SerializationRefResolver`, and rename the file from `ser_ref_resolver.dart` to `serialization_ref_resolver.dart`. 8. In `fury/lib/src/resolver/ref/deser_ref_resolver.dart`, rename `DeserRefResolver` to `DeserializationRefResolver`, and rename the file to `deserialization_ref_resolver.dart`. 9. Simplify the project structure by moving the contents of the `meta_str` and `ref` subdirectories in `fury/lib/src/resolver` to the `resolver` directory, and delete these two subdirectories. Also, move the contents of the `meta_str` and `ref` subdirectories in `fury/lib/src/resolver/impl` to the `resolver/impl` directory, and delete these two subdirectories. 10. Rename the file `fury-test/test/cross_lang_test/register_ser_test.dart` to `register_serialization_test.dart`. 11. Modify the default test group names: change the default name 'A group of tests' to specific names in `fury-test/test/cross_lang_test/register_ser_test.dart` and `fury-test/test/cross_lang_test/register_serialization_test.dart`.
|
@LouisLou2 Please add license header for all dart files: |
| * under the License. | ||
| */ | ||
|
|
||
| abstract class MsHandler{ |
There was a problem hiding this comment.
Could we avoid abbr here?
| import 'package:fury/src/meta/meta_string_byte.dart'; | ||
| import 'package:fury/src/resolver/ms_writing_resolver.dart'; | ||
|
|
||
| final class MsWritingResolverImpl extends MsWritingResolver{ |
There was a problem hiding this comment.
Could we avoid Ms abbr here?
| import 'package:fury/src/dev_annotation/optimize.dart'; | ||
| import 'package:fury/src/resolver/serialization_ref_resolver.dart'; | ||
|
|
||
| final class SerNoRefResolver extends SerializationRefResolver{ |
There was a problem hiding this comment.
Could we avoid Ser abbr here? Or we could remove Ser/Serialization directly
There was a problem hiding this comment.
Ser is not consistent with SerializationRefMeta naming in this class
1. Renamed MsHandler to MetaStringHandler in fury/lib/src/resolver/ms_handler.dart and renamed the file to meta_string_handler.dart. 2. Renamed MsWritingResolver to MetaStringWritingResolver in fury/lib/src/resolver/ms_writing_resolver.dart and renamed the file to meta_string_writing_resolver.dart. Also changed the method writeMsb to writeMetaStringBytes, and the local variable idenHash to identityHash within that method. 3. Renamed SerNoRefResolver to SerializationNoRefResolver in fury/lib/src/resolver/impl/ser_no_ref_resolver.dart and renamed the file to serialization_no_ref_resolver.dart. 4. Renamed DeserNoRefResolver to DeserializationNoRefResolver in fury/lib/src/resolver/impl/deser_no_ref_resolver.dart and renamed the file to deserialization_no_ref_resolver.dart. 5. Renamed SerMapRefResolver to SerializationMapRefResolver in fury/lib/src/resolver/impl/ser_map_ref_resolver.dart and renamed the file to serialization_map_ref_resolver.dart. 6. Renamed DeserMapRefResolver to DeserializationMapRefResolver in fury/lib/src/resolver/impl/deser_map_ref_resolver.dart and renamed the file to deserialization_map_ref_resolver.dart. 7. Changed the class name from SerPack to SerializerPack in fury/lib/src/serializer_pack.dart. 8. In fury/lib/src/resolver/tag_str_encode_resolver.dart, changed the encodeTn method to encodeTypeName, and updated all references in its subclasses accordingly. 9. In fury/lib/src/serializer/serializer_cache.dart, renamed getSer and getSerWithSpec to getSerializer and getSerializerWithSpec, also updating references in the subclasses. 10. Renamed the static fields _serDirector and _deserDirector to _serial in fury/lib/src/fury_impl.dart. 11. In fury/lib/src/serialize_coordinator.dart, renamed ser, _ser, serWithWriter to write, _write, _writeWithWriter respectively. 12. In fury/lib/src/serialize_coordinator.dart, renamed deser, _xDeser to read, _xRead respectively. 13. Renamed the file fury-test/test/cross_lang_test/cross_lang_struct_serial_test.dart to cross_lang_struct_serialization_test.dart. 14. Removed packages/fury/lib/src/codegen/analyze and fury/lib/src/codegen/collection/key/declare_simple_info.dart as they were unused. 15. Removed the fury/lib/src/codegen/collection/key folder and moved its files directly to the parent directory. 16. Removed the fury/lib/src/collection/key folder and moved its files directly to the parent directory. 17. In packages/fury/lib/src/serializer/array_serializer.dart, changed NumArraySerializer to NumericArraySerializer. 18. Removed the check for xlangMode in config (lines 59–64) in fury/lib/src/serializer/fury_header_serializer.dart.
chaokunyang
left a comment
chaokunyang
left a comment
There was a problem hiding this comment.
LGTM, awesome work!
2 participants
Add Fury Dart Support
Overview
This PR adds Dart language support to Apache Fury, implementing a comprehensive serialization solution for Dart and Flutter applications. Fury Dart consists of approximately 15,000 lines of code and provides an efficient serialization mechanism that works within Flutter's reflection limitations.
Implementation Approach
Dart supports reflection, but Flutter explicitly prohibits it. To address this constraint, Fury Dart uses a combination of:
This approach ensures compatibility with Flutter while maintaining the performance and flexibility expected from Fury.
Features
Usage Examples
Basic Class Serialization
After annotating your class with
@furyClass, run:This generates the necessary code in
some_class.g.dartand creates the_$SomeClassFurymixin.Serializing and Deserializing
Enum Serialization
Registration is similar to classes:
Type Support
Fury Dart currently supports the following type mappings in XLANG mode:
Project Structure
The implementation is organized into three main components:
SourceGen: Located at
dart-fury/packages/fury-core/lib/src/code_genHandles static code generation for serialization/deserialization.
FuryCore: Located at
dart-fury/packages/fury-core/lib/srcContains the core serialization and deserialization logic.
FuryTest: Located at
dart-fury/fury_testComprehensive test suite for Fury Dart functionality.
Testing Approach
The test suite is inspired by Fury Java's testing approach and includes:
Running Tests
Tests use the standard dart test framework.
To run tests:
Code Quality
Fury Dart maintains high code quality standards. You can verify this using:
Current Limitations
Development Information
Dependencies
fury-core package:
fury-test package:
Thank you for considering this PR. I'm available to address any questions or feedback regarding the implementation.