-
Notifications
You must be signed in to change notification settings - Fork 829
TINKERPOP-3166 Implement asNumber() step #3153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 3.8-dev
Are you sure you want to change the base?
Conversation
|
||
[gremlin-groovy,modern] | ||
---- | ||
g.inject(1).asNumber() <1> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would expect the basic happy path example to be included, something like g.inject("1234").asNumber()
.
This is cool. Is there a way this could be used in this format:
somewhere down the road? (This query might be a poor way to do this, I made it off the top of my head, really just wondering if this could be used in a way where someone could want to grab the top % of something and not have to count in 1 query then use that count and the percent to limit in the second) |
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/N.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Show resolved
Hide resolved
CHANGELOG.asciidoc
Outdated
@@ -76,6 +76,7 @@ This release also includes changes from <<release-3-7-XXX, 3.7.XXX>>. | |||
* Moved all lambda oriented Gremlin tests to `LambdaStepTest` in the Java test suite. | |||
* Removed the `@RemoteOnly` testing tag in Gherkin as lambda tests have all been moved to the Java test suite. | |||
* Updated gremlin-javascript to use GraphBinary as default instead of GraphSONv3 | |||
* Added the `asNumber()` steps to perform number conversion. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: "step" not "steps"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
new steps like this should probably come with an addition to the semantics doc.
@@ -342,6 +342,7 @@ public static List<TypeRegistration<?>> initV3Registrations() { | |||
add(GryoTypeReg.of(Pick.class, 137)); | |||
add(GryoTypeReg.of(DT.class, 198)); | |||
add(GryoTypeReg.of(Merge.class, 196)); | |||
add(GryoTypeReg.of(DT.class, 199)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
confused by this type declaration of DT
and the use of 199. is this meant for N
? and isn't the next number 205, not 199? and curious why serialization tests would be passing under this configuration....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh this looks like a miss from copying. And yea, I wonder why tests passed.
gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features/map/AsNumber.feature
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
.../src/test/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStepTest.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Outdated
Show resolved
Hide resolved
.../src/test/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStepTest.java
Outdated
Show resolved
Hide resolved
* Tokens that are used to denote different units of number. | ||
* Used with {@link AsNumberStep} step. | ||
*/ | ||
public enum N { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Radical idea at this point but does N
need to exist at all? It really doesn't serve any functional purpose that simply referencing the Number
class would not already do. would it be more clean to only define "N" where necessary? Like, the Grammar would need an N
to understand int
, float
, etc. and Javascript would need a way to reference BigDecimal
or whatever, but does Java (or .NET) itself need a redefinition of number types?
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/util/NumberHelper.java
Show resolved
Hide resolved
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/util/NumberHelper.java
Outdated
Show resolved
Hide resolved
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/util/NumberHelper.java
Outdated
Show resolved
Hide resolved
} | ||
if (!Double.isInfinite(a.doubleValue())) { | ||
// float losses precision, use string intermediate | ||
return a.getClass().equals(Float.class) ? Double.parseDouble(a.toString()) : a.doubleValue(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there loss of precision when directly using Float.doubleValue()?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By "loss" I mean Float.doubleValue() itself will add arbitrary decimal places when widening to double, since it's not an exact representation. e.g. "1.23" to float 1.23
becomes 1.2300000190734863
.
throw new ArithmeticException(String.format("Can't convert floating point infinity to %s.", numberToken)); | ||
} | ||
try { | ||
return num.getClass().equals(BigInteger.class) ? ((BigInteger) num).longValueExact() : num.longValue(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also use BigDecimal.longValueExact() as BigDecimal.longValue()
may result in some big numbers appearing small:
new BigDecimal(Long.MAX_VALUE+"1").longValue()
==>-9
gremlin> g.inject(new BigDecimal(Long.MAX_VALUE+"1")).asNumber(N.nshort)
==>-9
return a.intValue(); | ||
} | ||
} else if (clazz.equals(Long.class)) { | ||
return getLong(a, numberToken); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a strange inconsistency that arises by using long as the intermediate form for all of these integer types.
gremlin> g.inject(new Double(Integer.MAX_VALUE+"1")).asNumber(N.nint)
Can't convert number of type Double to Integer due to overflow.
Type ':help' or ':h' for help.
gremlin> g.inject(new Double(Long.MAX_VALUE+"1")).asNumber(N.nlong)
==>9223372036854775807
Double.longValue() does a primitive cast from double
to long
, which implicitly binds out-of-range values to Long.MAX_VALUE or Long.MIN_VALUE. Perhaps an extra check is needed in getLong() to throw an exception if the value exceeds the range of a long. The same concern would apply to float-to-long conversions.
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/util/NumberHelper.java
Outdated
Show resolved
Hide resolved
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/util/NumberHelper.java
Outdated
Show resolved
Hide resolved
...core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/AsNumberStep.java
Show resolved
Hide resolved
* @throws IllegalArgumentException if the specified numeric type is unsupported | ||
* @throws ArithmeticException if the number overflows | ||
*/ | ||
public static Number castTo(final Number a, final N numberToken) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an existing coerceTo
method which has similar casting logic (but instead of throwing on overflow, it will return the original type). I am wondering if the logic between coerceTo
and castTo
should be made consistent (can castTo
delegate most of the logic to coerceTo
?) and if it's a problem if they are inconsistent.
Co-authored-by: andreachild <andrea.child@improving.com>
…eTo and castTo to use the same conversion logic in NumberHelper. Removed `n` from N enum value in Grammar, and used `_` suffix for keyword collisions in Java/Groovy/JS.
} else if (clazz.equals(Float.class)) { | ||
// BigDecimal to double will overflow into Infinity, we want to handle this | ||
if (!a.getClass().equals(BigDecimal.class) && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BigInteger
also needs to be handled. Can add this unit test:
@Test(expected = ArithmeticException.class)
public void shouldOverflowIfBigDecimalCannotFitInFloat() {
NumberHelper.castTo(new BigDecimal("1E400"), N.float_);
}
@Test(expected = ArithmeticException.class)
public void shouldOverflowIfBigIntegerCannotFitInFloat() {
NumberHelper.castTo(new BigInteger("1").shiftLeft(2000), N.float_);
}
} else if (clazz.equals(Float.class)) { | ||
// BigDecimal to double will overflow into Infinity, we want to handle this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// BigDecimal to double will overflow into Infinity, we want to handle this | |
// BigDecimal or BigInteger to double can overflow into Infinity, we want to prevent this |
if (a.doubleValue() >= -Float.MAX_VALUE && a.doubleValue() <= Float.MAX_VALUE) { | ||
return a.floatValue(); | ||
} | ||
} else if (clazz.equals(Double.class)) { | ||
return a.doubleValue(); | ||
// BigDecimal to double will overflow into Infinity, we want to handle this | ||
if (!a.getClass().equals(BigDecimal.class) && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also need to handle BigInteger
. Can add these unit tests:
@Test(expected = ArithmeticException.class)
public void shouldOverflowIfBigDecimalCannotFitInDouble() {
NumberHelper.castTo(new BigDecimal("1E400"), N.double_);
}
@Test(expected = ArithmeticException.class)
public void shouldOverflowIfBigIntegerCannotFitInDouble() {
NumberHelper.castTo(new BigInteger("1").shiftLeft(2000), N.double_);
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same check is done for Float casting. Suggest to extract a common method for infinity/NaN check like:
private boolean isInfinityOrNaN(Number num) {
return (!num.getClass().equals(BigDecimal.class) && !num.getClass().equals(BigInteger.class) &&
(Double.isInfinite(num.doubleValue()) || Double.isNaN(num.doubleValue())));
}
final Integer value = 42; | ||
assertEquals(BigDecimal.valueOf(42), NumberHelper.castTo(value, N.bigDecimal)); | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest to add these test cases:
@Test
public void shouldCastDoubleInfinityToFloatInfinity() {
assertEquals(Float.POSITIVE_INFINITY, NumberHelper.castTo(Double.POSITIVE_INFINITY, N.float_));
assertEquals(Float.NEGATIVE_INFINITY, NumberHelper.castTo(Double.NEGATIVE_INFINITY, N.float_));
}
@Test
public void shouldCastFloatInfinityToDoubleInfinity() {
assertEquals(Double.POSITIVE_INFINITY, NumberHelper.castTo(Float.POSITIVE_INFINITY, N.double_));
assertEquals(Double.NEGATIVE_INFINITY, NumberHelper.castTo(Float.NEGATIVE_INFINITY, N.double_));
}
@Test
public void shouldCastFloatNaNToDoubleNaN() {
assertEquals(Double.NaN, NumberHelper.castTo(Float.NaN, N.double_));
}
@Test
public void shouldCastDoubleNaNToFloatNaN() {
assertEquals(Float.NaN, NumberHelper.castTo(Double.NaN, N.float_));
}
a.getClass().getSimpleName(), clazz.getSimpleName())); | ||
} | ||
|
||
private static Long getLong(final Number num, final Class<? extends Number> clazz) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
private static Long getLong(final Number num, final Class<? extends Number> clazz) { | |
private static Long longValue(final Number num, final Class<? extends Number> targetClass) { |
} | ||
String msg = String.format("Can't convert number of type %s to %s due to overflow.", | ||
num.getClass().getSimpleName(), clazz.getSimpleName()); | ||
if ((num.getClass().equals(Double.class) || num.getClass().equals(Float.class)) && (num.doubleValue()) > Long.MAX_VALUE) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we also need to check for underflow?
Finishing up the asNumber() step implementation following #3136.
This allows users to cast numbers into desired types, as well as to parse numerical string into desired numbers.
VOTE +1