Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
1df9d9b
feat(azure): implement Azure File Storage integration with basic file…
aryanmehrotra Sep 1, 2025
85f4030
feat(azure): enhance Azure File Storage integration with error handli…
aryanmehrotra Sep 1, 2025
b695504
test(azure): add comprehensive unit tests for Azure File Storage oper…
aryanmehrotra Sep 1, 2025
8f439f3
test(azure): add comprehensive unit tests for Azure File Storage oper…
aryanmehrotra Sep 1, 2025
307afe9
fixed merge conflicts
coolwednesday Nov 25, 2025
526e705
to be -reverted
coolwednesday Nov 25, 2025
00114c7
added zure file storage support
coolwednesday Nov 28, 2025
f2b1d1e
removed unnexessary files
coolwednesday Nov 28, 2025
faf2ea5
fixed linters
coolwednesday Nov 30, 2025
122f115
fixed linters and added tests
coolwednesday Dec 1, 2025
3a11b0f
added more info in docs
coolwednesday Dec 1, 2025
4ce0c46
minor fix
coolwednesday Dec 1, 2025
e94b304
Merge branch 'development' into en/azure
coolwednesday Dec 1, 2025
5f0022b
removed blob storage setup from contributing.md
coolwednesday Dec 2, 2025
76c48a4
Merge remote-tracking branch 'origin/en/azure' into en/azure
coolwednesday Dec 2, 2025
a366464
implemented review comments
coolwednesday Dec 2, 2025
4b5cf4e
Merge branch 'development' into en/azure
coolwednesday Dec 2, 2025
504320a
removed not required tests
coolwednesday Dec 2, 2025
01413eb
Merge remote-tracking branch 'origin/en/azure' into en/azure
coolwednesday Dec 2, 2025
b4f9860
add file storage Name in logs and metrics
coolwednesday Dec 2, 2025
1ffb682
removed unecessary info from contributing.md
coolwednesday Dec 3, 2025
5ba8b1d
Merge branch 'development' into en/azure
Umang01-hash Dec 3, 2025
2cd5724
Merge branch 'development' into en/azure
coolwednesday Dec 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 59 additions & 5 deletions docs/advanced-guide/handling-file/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ GoFr also supports FTP/SFTP file-store. Developers can also connect and use thei

- **AWS S3**
- **Google Cloud Storage (GCS)**
- **Azure File Storage**

The file-store can be initialized as follows:

Expand Down Expand Up @@ -163,6 +164,49 @@ func readFile(filename string) []byte {
> When using fake-gcs-server, authentication is not required.
> Currently supports one bucket per file-store instance.

### Azure File Storage as File-Store

Azure File Storage provides fully managed file shares in the cloud. To use Azure File Storage with GoFr:

```go
package main

import (
"gofr.dev/pkg/gofr"
"gofr.dev/pkg/gofr/datasource/file/azure"
)

func main() {
app := gofr.New()

// Create Azure File Storage filesystem
fs, err := azure.New(&azure.Config{
AccountName: "mystorageaccount",
AccountKey: "myaccountkey",
ShareName: "myshare",
// Endpoint is optional, defaults to https://{AccountName}.file.core.windows.net
// Endpoint: "https://custom-endpoint.file.core.windows.net",
}, app.Logger(), app.Metrics())

if err != nil {
app.Logger().Fatalf("Failed to initialize Azure File Storage: %v", err)
}

app.AddFileStore(fs)

app.Run()
}
```

> **Note:**
> - Azure File Storage uses file shares (similar to S3 buckets or GCS buckets)
> - Authentication requires both `AccountName` and `AccountKey`
> - The `Endpoint` field is optional and defaults to `https://{AccountName}.file.core.windows.net`
> - Currently supports one file share per file-store instance
> - The implementation automatically retries connection if the initial connection fails
> - **Automatic parent directory creation**: When creating files in nested paths (e.g., `dir1/subdir/file.txt`), parent directories are automatically created, matching local filesystem behavior
> - **Content type detection**: Content types are automatically detected based on file extensions (e.g., `.json` → `application/json`, `.txt` → `text/plain`)

### Creating Directory

To create a single directory
Expand Down Expand Up @@ -205,6 +249,7 @@ currentDir, err := ctx.File.Chdir("sub_dir")

> Note: This method attempts to change the directory, but S3's flat structure and fixed bucket
> make this operation inapplicable. Similarly, GCS uses a flat structure where directories are simulated through object prefixes.
> Azure File Storage supports directory operations natively, so `Chdir` works as expected.

### Read a Directory

Expand All @@ -227,7 +272,8 @@ for _, entry := range entries {
```

> Note: In S3 and GCS, directories are represented as prefixes of file keys/object names. This method retrieves file
> entries only from the immediate level within the specified directory.
> entries only from the immediate level within the specified directory. Azure File Storage supports native directory
> structures, so `ReadDir` works with actual directories.

### Creating and Save a File with Content

Expand All @@ -237,9 +283,14 @@ file, _ := ctx.File.Create("my_file.text")
_, _ = file.Write([]byte("Hello World!"))

// Closes and saves the file.
file.Close()
file.Close()
```

> **Note for Azure File Storage:**
> - Files can be created in nested directories (e.g., `dir1/subdir/file.txt`). Parent directories are automatically created if they don't exist
> - Content types are automatically detected based on file extensions (e.g., `.json`, `.txt`, `.csv`, `.xml`, `.html`, `.pdf`)
> - This behavior matches local filesystem operations for consistency

### Reading file as CSV/JSON/TEXT

GoFr support reading CSV/JSON/TEXT files line by line.
Expand Down Expand Up @@ -304,6 +355,8 @@ fmt.Printf("%v: %v Size: %v Last Modified Time : %v\n", entryType, entry.Name(),
> - Names starting with "0" are interpreted as binary files, with the "0" prefix removed (S3 specific behavior).
>
> For directories, the method calculates the total size of all contained objects and returns the most recent modification time. For files, it directly returns the file's size and last modified time.
>
> Azure File Storage supports native file and directory structures, so `Stat` returns accurate metadata for both files and directories.

### Rename/Move a File

Expand All @@ -319,7 +372,7 @@ err := ctx.File.Rename("old_name.text", "new_name.text")

`Remove` deletes a single file

> Note: Currently, the S3 package supports the deletion of unversioned files from general-purpose buckets only. Directory buckets and versioned files are not supported for deletion by this method. GCS supports deletion of both files and empty directories.
> Note: Currently, the S3 package supports the deletion of unversioned files from general-purpose buckets only. Directory buckets and versioned files are not supported for deletion by this method. GCS supports deletion of both files and empty directories. Azure File Storage supports deletion of both files and empty directories.

```go
err := ctx.File.Remove("my_dir")
Expand All @@ -328,13 +381,14 @@ err := ctx.File.Remove("my_dir")
The `RemoveAll` command deletes all subdirectories as well. If you delete the current working directory, such as "../currentDir", the working directory will be reset to its parent directory.

> Note: In S3, RemoveAll only supports deleting directories and will return an error if a file path (as indicated by a file extension) is provided for S3.
> GCS handles both files and directories.
> GCS and Azure File Storage handle both files and directories.

```go
err := ctx.File.RemoveAll("my_dir/my_text")
```

> GoFr supports relative paths, allowing locations to be referenced relative to the current working directory. However, since S3 and GCS use
> a flat file structure, all methods require a full path relative to the bucket.
> a flat file structure, all methods require a full path relative to the bucket. Azure File Storage supports native directory structures,
> so relative paths work as expected with directory navigation.

> Errors have been skipped in the example to focus on the core logic, it is recommended to handle all the errors.
1 change: 1 addition & 0 deletions go.work
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ use (
./pkg/gofr/datasource/dbresolver
./pkg/gofr/datasource/dgraph
./pkg/gofr/datasource/elasticsearch
./pkg/gofr/datasource/file/azure
./pkg/gofr/datasource/file/ftp
./pkg/gofr/datasource/file/s3
./pkg/gofr/datasource/file/gcs
Expand Down
Loading
Loading