Skip to content

TINKERPOP-3166 Implement asNumber() step #3153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: 3.8-dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ This release also includes changes from <<release-3-7-XXX, 3.7.XXX>>.
* Moved all lambda oriented Gremlin tests to `LambdaStepTest` in the Java test suite.
* Removed the `@RemoteOnly` testing tag in Gherkin as lambda tests have all been moved to the Java test suite.
* Updated gremlin-javascript to use GraphBinary as default instead of GraphSONv3
* Added the `asNumber()` step to perform number conversion.
* Renamed many types in the grammar for consistent use of terms "Literal", "Argument", and "Varargs"

== TinkerPop 3.7.0 (Gremfir Master of the Pan Flute)
Expand Down
27 changes: 27 additions & 0 deletions docs/src/dev/provider/gremlin-semantics.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -801,6 +801,33 @@ Incoming date remains unchanged.
See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsDateStep.java[source],
link:https://tinkerpop.apache.org/docs/x.y.z/reference/#asDate-step[reference]

[[asNumber-step]]
=== asNumber()

*Description:* converts the incoming traverser to the nearest parsable type if no argument is provided, or to the desired numerical type, based on the number token (`N`) provided.

*Syntax:* `asNumber()` | `asNumber(N numberToken)`

[width="100%",options="header"]
|=========================================================
|Start Step |Mid Step |Modulated |Domain |Range
|N |Y |N |`Number`/`String` |`Number`
|=========================================================

*Arguments:*

* `numberToken` - The enum `N` to denote the desired type to parse/cast to.

If no type token is provided, the incoming number remains unchanged.

*Exceptions*
* If any overflow occurs during narrowing of types, then an `ArithmeticException` will be thrown.
* If the incoming string cannot be parsed into a valid number format, then a `NumberFormatException` will be thrown.
* If the incoming traverser is a non-String/Number (including `null`) value then an `IllegalArgumentException` will be thrown.

See: link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java[source],
link:https://tinkerpop.apache.org/docs/x.y.z/reference/#asNumber-step[reference]

[[barrier-step]]
=== barrier()

Expand Down
31 changes: 31 additions & 0 deletions docs/src/reference/the-traversal.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -829,6 +829,37 @@ g.inject(datetime("2023-08-24T00:00:00Z")).asDate() <3>

link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#asDate()++[`asDate()`]

[[asNumber-step]]
=== AsNumber Step

The `asNumber()`-step (*map*) converts the incoming traverser to the nearest parsable type if no argument is provided, or to the desired numerical type, based on the number token (`N`) provided.

Numerical input will pass through unless a type is specified by the number token. `ArithmeticException` will be thrown for any overflow during narrowing of types.

String inputs are parsed into numeric values. By default, the value will be parsed as an integer if it represents a whole number, or as a double if it contains a decimal point. A `NumberFormatException` will be thrown if the string cannot be parsed into a valid number format.

All other input types will result in `IllegalArgumentException`.

[gremlin-groovy,modern]
----
g.inject(1234).asNumber() <1>
g.inject(1.76).asNumber() <2>
g.inject(1.76).asNumber(N.nint) <3>
g.inject("1b").asNumber() <4>
g.inject(33550336).asNumber(N.nbyte) <5>
----

<1> An int will be passed through.
<2> A double will be passed through.
<3> A double is converted into an int.
<4> String containing any character other than numerical ones will result in `NumberFormatException`.
<5> Narrowing of int to byte that overflows will throw `ArithmeticException`.

*Additional References*

link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#asNumber()++[`asNumber()`]
link:++https://tinkerpop.apache.org/javadocs/x.y.z/core/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.html#asNumber(org.apache.tinkerpop.gremlin.process.traversal.N)++[`asNumber(N)`]

[[barrier-step]]
=== Barrier Step

Expand Down
63 changes: 63 additions & 0 deletions docs/src/upgrade/release-3.8.x.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,69 @@ complete list of all the modifications that are part of this release.

=== Upgrading for Users

==== Number Conversion Step

We have been iterative introducing new language features into Gremlin, with the last major set of string, list and date manipulation
steps introduced in the 3.7 line. In 3.8.0, we are now introducing a number conversion step, `asNumber()`, to bridge
a gap in casting functionalities.

The new `asNumber()` serves as an umbrella step that parses strings and casts numbers into desired types. For the convenience of remote traversals in GLVs, these number types are denoted by a set of number tokens (`N`).

This new step will allow users to normalize their data by converting string numbers and mixed numeric types to consistent format, making it easier to perform downstream mathematical operations. As an example:

[source,text]
----
// sum() step can only take numbers
gremlin> g.inject(1.0, 2l, 3, "4", "0x5").sum()
class java.lang.String cannot be cast to class java.lang.Number

// use asNumber() to avoid casting exceptions
gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum()
==>15.0

// given sum() step returned a double, one can use asNumber() to further cast the result into desired type
gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum().asNumber(N.nint)
==>15
----

Semantically, the `asNumber()` step will convert the incoming traverser to the nearest parsable type if no argument is provided, or to the desired numerical type, based on the number token (`N`) provided.

Numerical input will pass through unless a type is specified by the number token. `ArithmeticException` will be thrown for any overflow during narrowing of types:

[source,text]
----
gremlin> g.inject(5.0).asNumber(N.nint)
==> 5 // casts double to int
gremlin> g.inject(12).asNumber(N.byte)
==> 12
gremlin> g.inject(128).asNumber(N.byte)
==> ArithmeticException
----

String input will be parsed. By default, the smalled unit of number to be parsed into is `int` if no number token is provided. `NumberFormatException` will be thrown for any unparsable strings:

[source,text]
----
gremlin> g.inject("5").asNumber()
==> 5
gremlin> g.inject("5.7").asNumber(N.int)
==> 5
gremlin> g.inject("1,000").asNumber(N.nint)
==> NumberFormatException
gremlin> g.inject("128").asNumber(N.nbyte)
==> ArithmeticException
----

All other input types will result in `IllegalArgumentException`:
[source,text]
----
gremlin> g.inject([1, 2, 3, 4]).asNumber()
==> IllegalArgumentException
----

See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#asNumber-step[asNumber()-step]
See: link:https://issues.apache.org/jira/browse/TINKERPOP-3166[TINKERPOP-3166]

==== Removal of Vertex/ReferenceVertex from grammar

`StructureVertex`, previously used to construct new vertices in the grammar, now been removed. The `V()` step, as well
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.IO;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Operator;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.P;
Expand Down Expand Up @@ -204,6 +205,7 @@ public final class CoreImports {
CLASS_IMPORTS.add(Direction.class);
CLASS_IMPORTS.add(DT.class);
CLASS_IMPORTS.add(Merge.class);
CLASS_IMPORTS.add(N.class);
CLASS_IMPORTS.add(Operator.class);
CLASS_IMPORTS.add(Order.class);
CLASS_IMPORTS.add(Pop.class);
Expand Down Expand Up @@ -363,6 +365,7 @@ public final class CoreImports {
Collections.addAll(ENUM_IMPORTS, Direction.values());
Collections.addAll(ENUM_IMPORTS, DT.values());
Collections.addAll(ENUM_IMPORTS, Merge.values());
Collections.addAll(ENUM_IMPORTS, N.values());
Collections.addAll(ENUM_IMPORTS, Operator.values());
Collections.addAll(ENUM_IMPORTS, Order.values());
Collections.addAll(ENUM_IMPORTS, Pop.values());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1067,6 +1067,14 @@ protected void notImplemented(final ParseTree ctx) {
* {@inheritDoc}
*/
@Override public T visitTraversalMethod_dateDiff_Date(final GremlinParser.TraversalMethod_dateDiff_DateContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
@Override public T visitTraversalMethod_asNumber_Empty(final GremlinParser.TraversalMethod_asNumber_EmptyContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
@Override public T visitTraversalMethod_asNumber_traversalN(final GremlinParser.TraversalMethod_asNumber_traversalNContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
Expand Down Expand Up @@ -1107,6 +1115,10 @@ protected void notImplemented(final ParseTree ctx) {
* {@inheritDoc}
*/
@Override public T visitTraversalDT(GremlinParser.TraversalDTContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
@Override public T visitTraversalN(GremlinParser.TraversalNContext ctx) { notImplemented(ctx); return null; }
/**
* {@inheritDoc}
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Operator;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.Pop;
Expand Down Expand Up @@ -2102,6 +2103,22 @@ public GraphTraversal visitTraversalMethod_dateDiff_Date(final GremlinParser.Tra
return graphTraversal.dateDiff(antlr.genericVisitor.parseDate(ctx.dateLiteral()));
}

/**
* {@inheritDoc}
*/
@Override
public GraphTraversal visitTraversalMethod_asNumber_Empty(final GremlinParser.TraversalMethod_asNumber_EmptyContext ctx) {
return graphTraversal.asNumber();
}

/**
* {@inheritDoc}
*/
@Override
public GraphTraversal visitTraversalMethod_asNumber_traversalN(final GremlinParser.TraversalMethod_asNumber_traversalNContext ctx) {
return graphTraversal.asNumber(
TraversalEnumParser.parseTraversalEnumFromContext(N.class, ctx.traversalN()));
}

public GraphTraversal[] getNestedTraversalList(final GremlinParser.NestedTraversalListContext ctx) {
return ctx.nestedTraversalExpr().nestedTraversal()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import org.apache.tinkerpop.gremlin.language.grammar.GremlinParser;
import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Operator;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.P;
Expand Down Expand Up @@ -207,6 +208,12 @@ public Void visitTraversalDT(final GremlinParser.TraversalDTContext ctx) {
return null;
}

@Override
public Void visitTraversalN(final GremlinParser.TraversalNContext ctx) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a few cases to GremlinTranslatorTest to ensure N is translating correctly?

appendExplicitNaming(ctx.getText(), N.class.getSimpleName());
return null;
}

@Override
public Void visitTraversalPredicate(final GremlinParser.TraversalPredicateContext ctx) {
switch(ctx.getChildCount()) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.tinkerpop.gremlin.process.traversal;

import org.apache.tinkerpop.gremlin.process.traversal.step.map.AsNumberStep;

import java.math.BigDecimal;
import java.math.BigInteger;

/**
* Tokens that are used to denote different units of number.
* Used with {@link AsNumberStep} step.
*/
public enum N {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Radical idea at this point but does N need to exist at all? It really doesn't serve any functional purpose that simply referencing the Number class would not already do. would it be more clean to only define "N" where necessary? Like, the Grammar would need an N to understand int, float, etc. and Javascript would need a way to reference BigDecimal or whatever, but does Java (or .NET) itself need a redefinition of number types?

nbyte(Byte.class),
nshort(Short.class),
nint(Integer.class),
nlong(Long.class),
nfloat(Float.class),
ndouble(Double.class),
nbigInt(BigInteger.class),
nbigDecimal(BigDecimal.class),;

private final Class<?> type;

N(Class<?> type) {this.type = type;}

public Class<?> getType() {
return this.type;
}

@Override
public String toString() {
return this.type.getSimpleName();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.Failure;
import org.apache.tinkerpop.gremlin.process.traversal.Merge;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.Order;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.process.traversal.Path;
Expand Down Expand Up @@ -83,6 +84,7 @@
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AddVertexStartStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AddVertexStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AsDateStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AsNumberStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AsStringGlobalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.AsStringLocalStep;
import org.apache.tinkerpop.gremlin.process.traversal.step.map.CallStep;
Expand Down Expand Up @@ -1915,6 +1917,30 @@ public default GraphTraversal<S, Long> dateDiff(final Traversal<?, ?> dateTraver
return this.asAdmin().addStep(new DateDiffStep<>(this.asAdmin(), dateTraversal));
}

/**
* Parse value of the incoming traverser as a {@link Number}.
*
* @return the traversal with an appended {@link AsNumberStep}.
* @see <a href="http://tinkerpop.apache.org/docs/${project.version}/reference/#AsNumberStep-step" target="_blank">Reference Documentation - AsNumberStep Step</a>
* @since 3.8.0
*/
public default GraphTraversal<S, Number> asNumber() {
this.asAdmin().getBytecode().addStep(Symbols.asNumber);
return this.asAdmin().addStep(new AsNumberStep<>(this.asAdmin()));
}

/**
* Parse value of the incoming traverser as a {@link Number}.
*
* @return the traversal with an appended {@link AsNumberStep}.
* @see <a href="http://tinkerpop.apache.org/docs/${project.version}/reference/#AsNumberStep-step" target="_blank">Reference Documentation - AsNumberStep Step</a>
* @since 3.8.0
*/
public default GraphTraversal<S, Number> asNumber(final N numberToken) {
this.asAdmin().getBytecode().addStep(Symbols.asNumber, numberToken);
return this.asAdmin().addStep(new AsNumberStep<>(this.asAdmin(), numberToken));
}

/**
* Calculates the difference between the list traverser and list argument.
*
Expand Down Expand Up @@ -4134,6 +4160,7 @@ private Symbols() {
public static final String asDate = "asDate";
public static final String dateAdd = "dateAdd";
public static final String dateDiff = "dateDiff";
public static final String asNumber = "asNumber";
public static final String all = "all";
public static final String any = "any";
public static final String merge = "merge";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
package org.apache.tinkerpop.gremlin.process.traversal.dsl.graph;

import org.apache.tinkerpop.gremlin.process.traversal.DT;
import org.apache.tinkerpop.gremlin.process.traversal.N;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.process.traversal.Path;
import org.apache.tinkerpop.gremlin.process.traversal.Pop;
Expand Down Expand Up @@ -809,6 +810,20 @@ public static <A> GraphTraversal<A, Long> dateDiff(final Traversal<?, OffsetDate
return __.<A>start().dateDiff(dateTraversal);
}

/**
* @see GraphTraversal#asNumber()
*/
public static <A> GraphTraversal<A, Number> asNumber() {
return __.<A>start().asNumber();
}

/**
* @see GraphTraversal#asNumber(N)
*/
public static <A> GraphTraversal<A, Number> asNumber(final N numberToken) {
return __.<A>start().asNumber(numberToken);
}

/**
* @see GraphTraversal#difference(Object)
*/
Expand Down
Loading
Loading