You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[reading Training Datasets](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/feature_view_api/#get_training_data)
15
-
-[creating In-Memory Training Datasets](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/feature_view_api/#training_data)
-[reading Training Datasets](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_view_api/#get_training_data)
15
+
-[creating In-Memory Training Datasets](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_view_api/#training_data)
| Feature Group Creation from dataframes | [`FeatureGroup.create_feature_group()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/feature_group_api/#create_feature_group) | :white_check_mark: | :white_check_mark: | - | - | - | Currently Flink/Beam/Java doesn't support registering feature group metadata.
24
+
| Feature Group Creation from dataframes | [`FeatureGroup.create_feature_group()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/feature_group_api/#create_feature_group) | :white_check_mark: | :white_check_mark: | - | - | - | Currently Flink/Beam/Java doesn't support registering feature group metadata.
25
25
Thus it needs to be pre-registered before you can write real time features computed by Flink/Beam. |
26
-
| Training Dataset Creation from dataframes |[`TrainingDataset.save()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/training_dataset_api/#save) |:white_check_mark:| - | - | - | - | Functionality was deprecated in version 3.0 |
27
-
| Data validation using Great Expectations for streaming dataframes |[`FeatureGroup.validate()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/feature_group_api/#validate) <br/> [`FeatureGroup.insert_stream()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/feature_group_api/#insert_stream) | - | - | - | - | - |`insert_stream` does not perform any data validation even when a expectation suite is attached. |
28
-
| Stream ingestion |[`FeatureGroup.insert_stream()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/feature_group_api/#insert_stream) |:white_check_mark:| - |:white_check_mark:|:white_check_mark:|:white_check_mark:| Python/Pandas/Polars has currently no notion of streaming. |
29
-
| Reading from Streaming Storage Connectors | [`KafkaConnector.read_stream()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/storage_connector_api/#read_stream) | :white_check_mark: | - | - | - | - | Python/Pandas/Polars has currently no notion of streaming.
26
+
| Training Dataset Creation from dataframes |[`TrainingDataset.save()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/training_dataset_api/#save)|:white_check_mark:| - | - | - | - | Functionality was deprecated in version 3.0 |
27
+
| Data validation using Great Expectations for streaming dataframes |[`FeatureGroup.validate()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/feature_group_api/#validate) <br/> [`FeatureGroup.insert_stream()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/feature_group_api/#insert_stream)| - | - | - | - | - |`insert_stream` does not perform any data validation even when a expectation suite is attached. |
28
+
| Stream ingestion |[`FeatureGroup.insert_stream()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/feature_group_api/#insert_stream)|:white_check_mark:| - |:white_check_mark:|:white_check_mark:|:white_check_mark:| Python/Pandas/Polars has currently no notion of streaming. |
29
+
| Reading from Streaming Storage Connectors | [`KafkaConnector.read_stream()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/storage_connector_api/#read_stream) | :white_check_mark: | - | - | - | - | Python/Pandas/Polars has currently no notion of streaming.
30
30
For Flink/Beam/Java only write operations are supported |
31
-
| Reading training data from external storage other than S3 |[`FeatureView.get_training_data()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/feature_view_api/#get_training_data) |:white_check_mark:| - | - | - | - | Reading training data that was written to external storage using a Storage Connector other than S3 can currently not be read using HSFS APIs, instead you will have to use the storage's native client. |
32
-
| Reading External Feature Groups into Dataframe |[`ExternalFeatureGroup.read()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/external_feature_group_api/#read) |:white_check_mark:| - | - | - | - | Reading an External Feature Group directly into a Pandas/Polars Dataframe is not supported, however, you can use the [Query API](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/query_api/) to create Feature Views/Training Data containing External Feature Groups. |
33
-
| Read Queries containing External Feature Groups into Dataframe |[`Query.read()`](<https://docs.hopsworks.ai/feature-store-api/{{{>hopsworks_version}}}/generated/api/query_api/#read) |:white_check_mark:| - | - | - | - | Reading a Query containing an External Feature Group directly into a Pandas/Polars Dataframe is not supported, however, you can use the Query to create Feature Views/Training Data and write the data to a Storage Connector, from where you can read up the data into a Pandas/Polars Dataframe. |
31
+
| Reading training data from external storage other than S3 |[`FeatureView.get_training_data()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/feature_view_api/#get_training_data)|:white_check_mark:| - | - | - | - | Reading training data that was written to external storage using a Storage Connector other than S3 can currently not be read using HSFS APIs, instead you will have to use the storage's native client. |
32
+
| Reading External Feature Groups into Dataframe |[`ExternalFeatureGroup.read()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/external_feature_group_api/#read)|:white_check_mark:| - | - | - | - | Reading an External Feature Group directly into a Pandas/Polars Dataframe is not supported, however, you can use the [Query API](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/query_api/) to create Feature Views/Training Data containing External Feature Groups. |
33
+
| Read Queries containing External Feature Groups into Dataframe |[`Query.read()`](https://docs.hopsworks.ai/feature-store-api/{{{hopsworks_version}}}/generated/api/query_api/#read)|:white_check_mark:| - | - | - | - | Reading a Query containing an External Feature Group directly into a Pandas/Polars Dataframe is not supported, however, you can use the Query to create Feature Views/Training Data and write the data to a Storage Connector, from where you can read up the data into a Pandas/Polars Dataframe. |
Copy file name to clipboardExpand all lines: docs/user_guides/fs/data_source/usage.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -160,4 +160,4 @@ Read more about training data creation [here](../feature_view/training-data.md).
160
160
## Next Steps
161
161
162
162
We have gone through the basic use cases of a data source.
163
-
For more details about the API functionality for any specific connector type, checkout the [API section](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/storage_connector_api/#storage-connector).
163
+
For more details about the API functionality for any specific connector type, checkout the [API section](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/storage_connector_api/#storage-connector).
Copy file name to clipboardExpand all lines: docs/user_guides/fs/feature_group/create.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,7 +39,7 @@ Using the HSFS API you can execute:
39
39
)
40
40
```
41
41
42
-
The full method documentation is available [here](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/feature_group_api/#featuregroup).
42
+
The full method documentation is available [here](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_group_api/#featuregroup).
43
43
If you need to create a feature group with vector similarity search supported, refer to [this guide](../vector_similarity_search.md#extending-feature-groups-with-similarity-search). `name` is the only mandatory parameter of the `create_feature_group` and represents the name of the feature group.
44
44
45
45
In the example above we created the first version of a feature group named *weather*, we provide a description to make it searchable to the other project members, as well as making the feature group available online.
@@ -352,7 +352,7 @@ If a feature group is online enabled, the `insert` method will store the feature
Copy file name to clipboardExpand all lines: docs/user_guides/fs/feature_group/create_external.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -77,7 +77,7 @@ Once you have defined the metadata, you can
77
77
fg.save()
78
78
```
79
79
80
-
The full method documentation is available [here](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/external_feature_group_api/#externalfeaturegroup). `name` is a mandatory parameter of the `create_external_feature_group` and represents the name of the feature group.
80
+
The full method documentation is available [here](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/external_feature_group_api/#externalfeaturegroup). `name` is a mandatory parameter of the `create_external_feature_group` and represents the name of the feature group.
81
81
82
82
The version number is optional, if you don't specify the version number the APIs will create a new version by default with a version number equals to the highest existing version number plus one.
83
83
@@ -125,18 +125,18 @@ For an external feature group to be available online, during the creation of the
125
125
```
126
126
127
127
The `insert()` method takes a DataFrame as parameter and writes it _only_ to the online feature store.
128
-
Users can select which subset of the feature group data they want to make available on the online feature store by using the [query APIs](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/query_api/).
128
+
Users can select which subset of the feature group data they want to make available on the online feature store by using the [query APIs](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/query_api/).
129
129
130
130
### Limitations
131
131
132
132
Hopsworks Feature Store does not support time-travel queries on external feature groups.
133
133
134
134
Additionally, support for `.read()` and `.show()` methods when using by the Python engine is limited to external feature groups defined on BigQuery and Snowflake and only when using the [Feature Query Service](../../../setup_installation/common/arrow_flight_duckdb.md).
135
-
Nevertheless, external feature groups defined top of any data source can be used to create a training dataset from a Python environment invoking one of the following methods: [create_training_data](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/feature_view_api/#create_training_data), [create_train_test_split](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/feature_view_api/#create_train_test_split) or the [create_train_validation_test_split](<https://docs.hopsworks.ai/hopsworks-api/{{{>hopsworks_version}}}/generated/api/feature_view_api/#create_train_validation_test_split)
135
+
Nevertheless, external feature groups defined top of any data source can be used to create a training dataset from a Python environment invoking one of the following methods: [create_training_data](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_view_api/#create_training_data), [create_train_test_split](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_view_api/#create_train_test_split) or the [create_train_validation_test_split](<https://docs.hopsworks.ai/hopsworks-api/{{{hopsworks_version}}}/generated/api/feature_view_api/#create_train_validation_test_split)
0 commit comments