You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+24-10Lines changed: 24 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -70,7 +70,7 @@ Put all data files to be uploaded in the same folder.
70
70
### Prepare manifest
71
71
72
72
A manifest is a metadata (TSV) file that contains information about the data files to be uploaded. CLI tool will use this information to find, validate, and upload data files to Data Hub. There are 4 columns that are relevant to CLI tool:
73
-
- the column containing file IDs
73
+
- the column containing file IDs, unless you are updating existing data files and know the correct file IDs, manifest should not include this column, so that CLI will generate this column automatically.
74
74
- the column containing file names
75
75
- the column containing file sizes
76
76
- the column containing file MD5 checksums
@@ -80,7 +80,7 @@ Different Data Commons may have different column names, but they all contain the
80
80
You can put a manifest in the same folder with the data files, or you can put it in a separate folder.
81
81
82
82
### Prepare configuration file
83
-
- Make a copy of the example config file: “crdc-datahub-cli-uploader/configs/uploader-file-config.example.yml”, give it an appropriate name, in this document we name it “file-upload.yml”
83
+
- Make a copy of the example config file: “configs/uploader-file-config.example.yml”, give it an appropriate name, in this document we name it “file-upload.yml”
84
84
- Open the new config file with a text editor, preferably a code editor like Sublime Text, Notepad++, VSCode, Vim, Emacs etc. Please DO NOT use a word processor like Word or Pages to open the config file.
85
85
- Configurations are in “key: value” format. There must be a space between colon and the value.
86
86
- api-url: keep it unchanged, unless you are using an environment other than Data Hub production environment
@@ -106,8 +106,19 @@ You do not have to upload the this final manifest in the CRDC submission portal.
106
106
107
107
If you need to update the content of the file manifest, then edit the final manifest and upload it through the CRDC submission portal.
108
108
If you need to upload the data files again, then you can use the final manifest with the Uploader CLI tool. The Uploader CLI Tool will use the file IDs/Keys provided in this file manifest instead of generating new ones.
109
+
109
110
### Execute upload command
110
111
112
+
For **macOS binary**:
113
+
114
+
`$ ./uploader --config configs/file-upload.yml`
115
+
116
+
For **Windows binary**:
117
+
118
+
`> uploader.exe --config configs/file-upload.yml`
119
+
120
+
For **Source code**:
121
+
111
122
Depends on how Python3 was installed, on some systems you need to use “python” instead of “python3” in following command.
@@ -119,7 +130,7 @@ Depends on how Python3 was installed, on some systems you need to use “python
119
130
Put all metadata (TSV) files to be uploaded in the same folder.
120
131
121
132
### Prepare configuration file
122
-
- Make a copy of the example config file: “crdc-datahub-cli-uploader/configs/uploader-metadata-config.example.yml”, give it an appropriate name, in this document we name it “metadata-upload.yml”
133
+
- Make a copy of the example config file: “configs/uploader-metadata-config.example.yml”, give it an appropriate name, in this document we name it “metadata-upload.yml”
123
134
- Open the new config file with a text editor, preferably a code editor like Sublime Text, Notepad++, VSCode, Vim, Emacs etc. Please DO NOT use a word processor like Word or Pages to open the config file.
124
135
- Configurations are in “key: value” format. There must be a space between colon and the value.
125
136
- api-url: keep it unchanged, unless you are using an environment other than Data Hub production environment
@@ -133,15 +144,18 @@ Put all metadata (TSV) files to be uploaded in the same folder.
133
144
134
145
### Execute upload command
135
146
136
-
Depends on how Python3 was installed, on some systems you need to use “python” instead of “python3” in following command.
0 commit comments