Skip to content

adlfs authentication changed? Old code is suddenly breaking. #496

@yingtho

Description

@yingtho

The code below worked fine until the 5/5/2025 (this code only runs once a week) and then suddenly show error:
The log

ValueError                                Traceback (most recent call last)
File ~/cluster-env/clonedenv/lib/python3.10/site-packages/adlfs/spec.py:581, in AzureBlobFileSystem.do_connect(self)
    580     else:
--> 581         raise ValueError(
    582             "Must provide either a connection_string or account_name with credentials!!"

The code:

import fsspec
import json
from io  import BytesIO
from py4j.protocol import Py4JJavaError
from lxml import etree
import xml.etree.ElementTree as ET 

rowTags = json.loads(RowTags)

rowTags_lower_case_first_letter = list(map(lambda s: s[0].lower() + s[1:], rowTags))

path = mssparkutils.fs.ls(f"abfss://{SourceContainer}@{SourceFilesystem}.dfs.core.windows.net/raw/{TableDirectory}/")[0].path

header = ""
with fsspec.open(path, 'r') as file:
    for line in file:
        if '<items>' in line:
            items_start = line.encode()
            tail = """  </items>
</export>"""
            break
        else:
            header += line

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions