Skip to content

Dynamic Python class initialization error when loading multiple Dynamic Pkl objects #11

@tianda-huang-ai

Description

@tianda-huang-ai

The following is on Python 3.10 and pkl-python 0.1.16.

Two different Dynamic objects with different members, e.g.

dynamic1 {
    a = "a"
}
dynamic2 {
    b = "b"
    c = "c"
}

causes the following error upon attempting to load:

>>> import pkl
>>> pkl.__version__
'0.1.16'
>>> pkl.loads("""
... dynamic1 {
...     a = "a"
... }
... dynamic2 {
...     b = "b"
...     c = "c"
... }
... """)

-----------------------------------------------------------------------------------------------------------------------
<stdin> 1 <module>
1

__init__.py 112 loads
return load(

__init__.py 93 load
config = evaluator.evaluate_expression(source, expr)

evaluator_manager.py 86 evaluate_expression
parsed = self.parser.parse(decoded)

parser.py 126 parse
return self.handle_type(obj)

parser.py 132 handle_type
return self.type_handlers[type_code](obj)

parser.py 155 parse_typed_dynamic
property_list = list(map(self.handle_type, members))

parser.py 132 handle_type
return self.type_handlers[type_code](obj)

parser.py 221 parse_property
return {key: self.handle_type(value)}

parser.py 132 handle_type
return self.type_handlers[type_code](obj)

parser.py 177 parse_typed_dynamic
res = clazz(*members.values())

TypeError:
Dynamic.__init__() takes 2 positional arguments but 3 were given

The error seems to be due to pulling the first Dynamic instance from the dataclass_cache on subsequent calls to get_dataclass_class -- I can get it to load with the following changes, but I'm not sure if there's anything else depending on having the cache to work correctly.

diff --git src/pkl/parser.py src/pkl/parser.py
index 7e7f0f1..b60a623 100644
--- src/pkl/parser.py
+++ src/pkl/parser.py
@@ -139,8 +139,8 @@ class Parser:
             return type(obj)(self.handle_type(v) for v in obj)
         return obj
 
-    def get_dataclass_class(self, class_name: str, keys: List[str]):
-        if class_name not in self.dataclass_cache:
+    def get_dataclass_class(self, class_name: str, keys: List[str], no_cache=False):
+        if no_cache or class_name not in self.dataclass_cache:
             dynamic_class = make_dataclass(class_name, keys)
             dynamic_class.__module__ = _lookup_module.__name__
             setattr(_lookup_module, class_name, dynamic_class)
@@ -173,7 +173,8 @@ class Parser:
                 raise ValueError(f"'namespace' provided but '{class_name}' not found")
             clazz = self.namespace[class_name]
         else:
-            clazz = self.get_dataclass_class(class_name, members.keys())
+            no_cache = (class_name == "Dynamic")
+            clazz = self.get_dataclass_class(class_name, members.keys(), no_cache)
         res = clazz(*members.values())
         return res

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions