Billing details not populating on UI #1821
Description
Hi,
I have been running a Datalab instance but am not seeing any data under the billing section in the front end. AWS is definitely incurring costs as a result of Datalab ($40 for the month of June).
Is there some specific condition that needs to be fulfilled before billing is populated?
This is the command I used to create the Datalab:
/usr/bin/python3 ~/incubator-datalab/infrastructure-provisioning/scripts/deploy_datalab.py
--conf_service_base_name datalab-base-name
--conf_tag_resource_id datalab-resource-id
--conf_os_family debian
--key_path /home/ubuntu/.ssh/
--conf_key_name datalab
--action create
--keycloak_realm_name master
--keycloak_user XXXXXXXX
--keycloak_user_password XXXXXXXX
--keycloak_auth_server_url http://XX.XX.XXX.XX:8080
'aws'
--aws_region eu-west-1
--aws_zone eu-west-1a
--aws_ssn_instance_size t2.medium
--aws_billing_bucket datalabbilling
--aws_account_id XXXXXXXXXXX
--aws_access_key XXXXXXXXXXXXXXX
--aws_secret_access_key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
An excerpt from the logs in /var/opt/datalab/log/ssn/billing.log:
2023-07-03 11:25:30.107 INFO 94517 --- [cluster-ClusterId{value='64a2b029b4d0cc7135f64a6e', description='null'}-localhost:27017] org.mongodb.driver.cluster : Discovered cluster type of STANDALONE
2023-07-03 11:25:30.919 INFO 94517 --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8088 (https) with context path '/api/billing'
2023-07-03 11:25:30.922 INFO 94517 --- [main] com.epam.datalab.BillingAwsApplication : Started BillingAwsApplication in 21.602 seconds (JVM running for 23.418)
2023-07-03 11:25:30.926 DEBUG 94517 --- [main] com.epam.datalab.BillingServiceImpl : Billing report configuration file: /opt/datalab/conf/billing.yml
INFO [2023-07-03 11:30:00,751] org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/api/billing]: Initializing Spring DispatcherServlet 'dispatcherServlet'
INFO [2023-07-03 11:30:00,751] org.springframework.web.servlet.DispatcherServlet: Initializing Servlet 'dispatcherServlet'
INFO [2023-07-03 11:30:00,760] org.springframework.web.servlet.DispatcherServlet: Completed initialization in 9 ms
DEBUG [2023-07-03 11:30:01,109] com.epam.datalab.module.aws.AdapterS3File: Adapter S3 will be opened for READ
DEBUG [2023-07-03 11:30:02,317] com.epam.datalab.module.aws.AdapterS3File: New report files in bucket folder datalabbilling not found
DEBUG [2023-07-03 11:30:02,318] com.epam.datalab.module.aws.AdapterS3File: Adapter S3 has been opened
DEBUG [2023-07-03 11:30:02,318] com.epam.datalab.core.parser.ParserByLine: Source data has multy entry true
DEBUG [2023-07-03 11:45:00,243] com.epam.datalab.module.aws.AdapterS3File: Adapter S3 will be opened for READ
DEBUG [2023-07-03 11:45:00,332] com.epam.datalab.module.aws.AdapterS3File: New report files in bucket folder datalabbilling not found
DEBUG [2023-07-03 11:45:00,332] com.epam.datalab.module.aws.AdapterS3File: Adapter S3 has been opened
DEBUG [2023-07-03 11:45:00,332] com.epam.datalab.core.parser.ParserByLine: Source data has multy entry true
Excerpt from the logs in /var/opt/datalab/log/ssn/selfservice.log:
INFO [2023-07-03 11:25:30,534] org.eclipse.jetty.server.handler.ContextHandler: Started i.d.j.MutableServletContextHandler@5a654e05{/,null,AVAILABLE}
INFO [2023-07-03 11:25:30,559] org.eclipse.jetty.server.AbstractConnector: Started application@359e27d2{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
INFO [2023-07-03 11:25:30,565] org.eclipse.jetty.server.AbstractConnector: Started admin@277bc3a5{SSL,[ssl, http/1.1]}{0.0.0.0:8444}
INFO [2023-07-03 11:25:30,565] org.eclipse.jetty.server.Server: Started @23057ms
INFO [2023-07-03 11:25:30,616] com.epam.datalab.backendapi.dropwizard.listeners.MongoStartupListener: Populating DataLab default roles into database
INFO [2023-07-03 11:25:30,648] com.epam.datalab.backendapi.dropwizard.listeners.MongoStartupListener: Check for connected endpoints:
connected endpoints: 1
connected clouds: [AWS]
INFO [2023-07-03 11:30:00,075] com.epam.datalab.backendapi.schedulers.CheckInfrastructureStatusScheduler: Trying to update infrastructure statuses
INFO [2023-07-03 11:30:00,194] com.epam.datalab.backendapi.service.impl.InfrastructureInfoServiceImpl: EnvResources is empty: EnvResourceList{host=[], cluster=[]} , didn't send request to provisioning service
INFO [2023-07-03 11:30:00,208] com.epam.datalab.backendapi.schedulers.billing.BillingScheduler: Trying to update billing
INFO [2023-07-03 11:30:02,580] com.epam.datalab.backendapi.service.impl.BillingServiceImpl: Updating billing information for endpoint local. Billing data []
INFO [2023-07-03 11:45:00,064] com.epam.datalab.backendapi.schedulers.CheckInfrastructureStatusScheduler: Trying to update infrastructure statuses
INFO [2023-07-03 11:45:00,094] com.epam.datalab.backendapi.service.impl.InfrastructureInfoServiceImpl: EnvResources is empty: EnvResourceList{host=[], cluster=[]} , didn't send request to provisioning service