-
Notifications
You must be signed in to change notification settings - Fork 109
Open
Description
The code below worked fine until the 5/5/2025 (this code only runs once a week) and then suddenly show error:
The log
ValueError Traceback (most recent call last)
File ~/cluster-env/clonedenv/lib/python3.10/site-packages/adlfs/spec.py:581, in AzureBlobFileSystem.do_connect(self)
580 else:
--> 581 raise ValueError(
582 "Must provide either a connection_string or account_name with credentials!!"
The code:
import fsspec
import json
from io import BytesIO
from py4j.protocol import Py4JJavaError
from lxml import etree
import xml.etree.ElementTree as ET
rowTags = json.loads(RowTags)
rowTags_lower_case_first_letter = list(map(lambda s: s[0].lower() + s[1:], rowTags))
path = mssparkutils.fs.ls(f"abfss://{SourceContainer}@{SourceFilesystem}.dfs.core.windows.net/raw/{TableDirectory}/")[0].path
header = ""
with fsspec.open(path, 'r') as file:
for line in file:
if '<items>' in line:
items_start = line.encode()
tail = """ </items>
</export>"""
break
else:
header += line
Metadata
Metadata
Assignees
Labels
No labels