Web UI setup for 0.0.0.0 or differen machine

[MyraBaba]

We can succesfully make search on old legacy search UI.

We installed all at remote linux and trying to reach from outside.

BUT:

New modern Web UI never worked. showed archive but never make search and show results.

Is there a clear setup information for WEB UI to operate remote ip _

[junhaoliao]

Hi @MyraBaba ,

Thank you for trying out the new release and reaching out!

To help us debug the issue, could you please provide the following information here in the discussion thread? (Alternatively, you can email it to us at info@yscope.com .)

1. The updated clp-package/etc/clp-config.yml.
2. A brief description of how the network is set up and accessed. For example, did you start the package on one server and access it from another machine using its IP address? Was a reverse proxy (e.g., Nginx) or firewall involved?

(Assuming a search request has just been submitted via the Web UI)
3. Output of docker logs $(docker ps -aqf "name=^clp-webui") for the webui logs.
4. The query scheduler logs at clp-package/var/log/<hostname>/query_scheduler/query_scheduler.log

In the meantime, could you try using our command-line tools to see if the issue persists?
clp-json: https://docs.yscope.com/clp/main/user-guide/quick-start/clp-json.html#searching-from-the-command-line
clp-text: https://docs.yscope.com/clp/main/user-guide/quick-start/clp-text.html#searching-from-the-command-line

If neither works, please try restarting the package and see if the issue resolves:

sbin/stop-clp.sh
sbin/start-clp.sh

Let us know how it goes!

Thanks,
Junhao

[MyraBaba]

Command lne is working.
Is there any clean steps for :

setup CLP serverstool (both from docker and package)

setup modern WEB UI running from both binded 0.0.0.0 (reachable from different ip/machine ) and also running ON sepearate machine than CLP

[hoophalab]

Thanks for the update!

1. Regarding the Web UI setup, we tested the Web UI on a few different environments (bare metal Linux, a Linux VM, and WSL) and found that 0.0.0.0 was always reachable from another machine, but maybe we're not running it the same way as you. Could you check the following:

   a. Are you running the Web UI directly? If so, could you confirm you're running the client with npm start -- --host 0.0.0.0 in the components/webui/client directory?

   b. If you're running the Web UI in Docker, could you check that your reverse proxy or port forward is set up correctly?

2. For running the web UI on a separate machine, our multi-node setup guide covers how to do that: https://docs.yscope.com/clp/main/user-guide/guides-multi-node.html

If you're still running into different issues after trying these steps, could you join our Discord server? Then we can join a call together to try and debug things.

Let us know if you have any other questions!

[MyraBaba]

Sorry for delay , Now we have time today to try again CLP to make search over forensic data , logs. We want to use modern UI . Would mind to give me full steps to install linux / ubuntu nd index uploaded files . PS: can we sepearet search file/folder per icident? Best

…

On 4 Aug 2025, at 18:50, hoophalab ***@***.***> wrote: Thanks for the update! Regarding the Web UI setup, we tested the Web UI on a few different environments (bare metal Linux, a Linux VM, and WSL) and found that 0.0.0.0 was always reachable from another machine, but maybe we're not running it the same way as you. Could you check the following: a. Are you running the Web UI directly? If so, could you confirm you're running the client with npm start -- --host 0.0.0.0 in the components/webui/client directory? b. If you're running the Web UI in Docker, could you check that your reverse proxy or port forward is set up correctly? For running the web UI on a separate machine, our multi-node setup guide covers how to do that: https://docs.yscope.com/clp/main/user-guide/guides-multi-node.html If you're still running into different issues after trying these steps, could you join our Discord server <https://discord.com/invite/7kZA2m5G87>? Then we can join a call together to try and debug things. Let us know if you have any other questions! — Reply to this email directly, view it on GitHub <#1135 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZH76Z67Y5M7JLSCFMT33L56LVAVCNFSM6AAAAACCTZU3OSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGOJZGEYDAMA>. You are receiving this because you were mentioned.

[hoophalab]

Thank you for experimenting with the new release and reaching out!

To install Ubuntu with a desktop environment, you can follow their detailed official guide. After installing the operating system, you can follow these steps to experiment with CLP.

1. Install docker
   sudo apt install docker.io

2. Download the latest clp release
   https://github.com/y-scope/clp/releases/download/v0.5.0/clp-text-x86_64-v0.5.0.tar.gz

3. Extract the compressed file
   tar xf clp-text-x86_64-v0.5.0.tar.gz

4. Edit the configuration file for remote access
   To access the WebUI from a separate machine, you need to edit clp-json-x86_64-v0.5.0/etc/clp-config.yml. Uncomment the following sections and replace localhost with 0.0.0.0.

From:

#webui:
#  host: "localhost"
#  port: 4000
#  results_metadata_collection_name: "results-metadata"

To:

webui:
  host: "0.0.0.0"
  port: 4000
  results_metadata_collection_name: "results-metadata"

5. Start CLP in single-node mode
   sudo clp-json-x86_64-v0.5.0/sbin/start-clp.sh
   When the script completes, the WebUI can be accessed via localhost:4000 on the server or <server_ip>:4000 from another machine.

6. To compress a log file, you can run the following command on the server.
   sudo clp-json-x86_64-v0.5.0/sbin/compress.sh <log_file_path>

Let us know how it goes! Our team are actively working to simplify deployments through Docker Compose and Kubernetes.
You can follow progress and updates on GitHub:
Docker Compose (ETA: Sept 26) → Issue #1177 → PR #1178
Kubernetes (ETA: end of October) → Issue #1309

[MyraBaba]

Thanks is it better to use clp-json ot clp-text ?

…

On 18 Sep 2025, at 20:02, hoophalab ***@***.***> wrote: https://github.com/y-scope/clp/releases/download/v0.5.0/clp-text-x86_64-v0.5.0.tar.gz

[junhaoliao]

Hello,

clp-json is for JSON logs and clp-text is unstructured text logs. Both of them share the same core libraries but are configured differently.

You may use clp-json if your logs are structured as independent JSON objects (e.g., MongoDB logs), as it allows field-specific searches. For example,

{"timestamp": 0, "level": "INFO", "message": "Hello world", "success": true}
{"timestamp": 1, "level": "DEBUG", "message": "Retrying...", "latency_ms": 152.4}
{"timestamp": 2, "level": "ERROR", "message": "Connection failed", "retries": 3}

With this format, you could search for all events where success: true, or retries > 2, or latency_ms < 200 AND level: DEBUG.

---

You may use clp-text for plain text logs where each event may span multiple lines (e.g., syslogs), using wildcard-based searches. For example,

Sep 19 10:15:42 INFO myhost app[111]: request_id=abc123 success=true duration=152ms
Sep 19 10:15:45 DEBUG myhost app[112]: request_id=def456 success=false retries=3
Sep 19 10:15:47 ERROR myhost kernel: [123456.789012] eth0: link is up

In this case, the fields like success=true and retries=3 are just text; you can only search them textually (e.g., search for success=false or retries=3), not with numeric or boolean comparisons.

---

In the future, we plan to unify the package types into a single package, so users can seamlessly work with both JSON and text logs without additional selection or configuration. Stay tuned for updates!

Let us know if you have any questions!

Thanks,

Junhao

[MyraBaba]

So converting logs to json and than index / search has better search results and options ?

…

On 20 Sep 2025, at 03:48, Junhao Liao ***@***.***> wrote: Hello, clp-json is for JSON logs and clp-text is unstructured text logs. Both of them share the same core libraries but are configured differently. You may use clp-json if your logs are structured as independent JSON objects (e.g., MongoDB logs), as it allows field-specific searches. For example, {"timestamp": 0, "level": "INFO", "message": "Hello world", "success": true} {"timestamp": 1, "level": "DEBUG", "message": "Retrying...", "latency_ms": 152.4} {"timestamp": 2, "level": "ERROR", "message": "Connection failed", "retries": 3} With this format, you could search for all events where success: true, or retries > 2, or latency_ms < 200 AND level: DEBUG. You may use clp-text for plain text logs where each event may span multiple lines (e.g., syslogs), using wildcard-based searches. For example, Sep 19 10:15:42 INFO myhost app[111]: request_id=abc123 success=true duration=152ms Sep 19 10:15:45 DEBUG myhost app[112]: request_id=def456 success=false retries=3 Sep 19 10:15:47 ERROR myhost kernel: [123456.789012] eth0: link is up In this case, the fields like success=true and retries=3 are just text; you can only search them textually (e.g., search for success=false or retries=3), not with numeric or boolean comparisons. In the future, we plan to unify the package types into a single package, so users can seamlessly work with both JSON and text logs without additional selection or configuration. Stay tuned for updates! Let us know if you have any questions! Thanks, Junhao — Reply to this email directly, view it on GitHub <#1135 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZH5AJD6JZZCDWK2RM233TSP5PAVCNFSM6AAAAACCTZU3OSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINBWGE4DGOA>. You are receiving this because you were mentioned.

[junhaoliao]

If by “converting” you mean taking text logs from your application, parsing them (e.g., with regex) into structured fields, and then writing JSON logs, the benefits really depend on your log volume — parsing can introduce significant overhead.

It’s generally recommended to choose the CLP package that matches your logs’ original format.

While querying JSON logs is more flexible (numeric, boolean, and field-based searches), equivalent wildcard queries for text logs often work well when you only need string matches. On the other hand, if your application can be configured to natively output JSON logs instead of text logs, conversion isn’t necessary.

If you’d like, you can share some sample logs and/or the types of the applications here (or, if privacy is a concern, to info@yscope.com), and we’ll be happy to analyze them and provide further recommendations.

[MyraBaba]

I attached both original css (exported from $MFT) and converted to json. I indexed jsonl with your given command . It says %95 saving.. but can’t search and find nothing. I believe I am searching wrong. Attached example 300 lines Also we have dates. Want to make search limit by both of them in same file. Other enterprise questions but lets first have somthin working. Best 

…

On 20 Sep 2025, at 13:17, Junhao Liao ***@***.***> wrote: If by “converting” you mean taking text logs from your application, parsing them (e.g., with regex) into structured fields, and then writing JSON logs, the benefits really depend on your log volume — parsing can introduce significant overhead. It’s generally recommended to choose the CLP package that matches your logs’ original format. While querying JSON logs is more flexible (numeric, boolean, and field-based searches), equivalent wildcard queries for text logs often work well when you only need string matches. On the other hand, if your application can be configured to natively output JSON logs instead of text logs, conversion isn’t necessary. If you’d like, you can share some sample logs and/or the types of the applications here (or, if privacy is a concern, to ***@***.*** ***@***.***>), and we’ll be happy to analyze them and provide further recommendations. — Reply to this email directly, view it on GitHub <#1135 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZH4RUDWYVS5C6WSOSAD3TUSU7AVCNFSM6AAAAACCTZU3OSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINBWGM2DAMI>. You are receiving this because you were mentioned.

[hoophalab]

Hi,

We can't see the attached logs in the GitHub comment. There are two  characters, but we're unable to download them.

Based on your description, we think the issue might be that you forgot to specify --timestamp-key when compressing the logs, as the documentation requires.
sbin/compress.sh --timestamp-key '<timestamp-key>' <path1> [<path2> ...]

We understand that this is a usability issue since we can't guarantee that every log has a timestamp. However, our search methods were initially built around logs that do have them. We're actively looking into ways to fix this.

[MyraBaba]

Thank you for sharing the CLP project details. We're currently conducting performance benchmarking to validate CLP's speed advantages over existing solutions like Splunk and other market leaders. If our testing confirms superior performance metrics, this represents a significant market opportunity. My diverse background spanning low-level C development, hardware engineering, enterprise architecture, field operations, customer implementation, project management, and executive leadership—including current board positions—provides me with multiple perspectives on both the technical merit and commercial viability of this solution. I'm particularly interested in understanding how CLP's performance translates to real-world enterprise scenarios and the potential competitive differentiation this could create in the log processing market. I'd welcome the opportunity to discuss the technical benchmarks and market positioning strategy in more detail. Best regards, BC

…

On 22 Sep 2025, at 23:27, hoophalab ***@***.***> wrote: Hi, We can't see the attached logs in the GitHub comment. There are two  characters, but we're unable to download them. Based on your description, we think the issue might be that you forgot to specify --timestamp-key when compressing the logs, as the documentation <https://docs.yscope.com/clp/main/user-docs/quick-start/clp-json.html#compressing-json-logs> requires. sbin/compress.sh --timestamp-key '<timestamp-key>' <path1> [<path2> ...] We understand that this is a usability issue since we can't guarantee that every log has a timestamp. However, our search methods <https://www.usenix.org/system/files/osdi24-wang-rui.pdf> were initially built around logs that do have them. We're actively looking into ways to fix this. — Reply to this email directly, view it on GitHub <#1135 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZH2GUID2Z6YJO27ZZDL3UBLSTAVCNFSM6AAAAACCTZU3OSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINBYGA2DOOI>. You are receiving this because you were mentioned.

[hoophalab]

We're glad to hear you're conducting your own performance benchmarks and we're ready to provide any assistance you need.

Our system has been tested and proven to be reliable and efficient, handling more than 200TB of log data per day for Uber. You can check their blogs for detail:
https://www.uber.com/en-CA/blog/reducing-logging-cost-by-two-orders-of-magnitude-using-clp/
https://www.uber.com/en-CA/blog/modernizing-logging-with-clp-ii/

We'd be happy to discuss collaboration opportunities and explore how CLP can address your specific use case.

Best regards

[MyraBaba]

Thanks for the update. Is there any recent use cases ? Also same data head to head splunk / clptest results ?

…

On 23 Sep 2025, at 19:03, hoophalab ***@***.***> wrote: We're glad to hear you're conducting your own performance benchmarks and we're ready to provide any assistance you need. Our system has been tested and proven to be reliable and efficient, handling more than 200TB of log data per day for Uber. You can check their blogs for detail: https://www.uber.com/en-CA/blog/reducing-logging-cost-by-two-orders-of-magnitude-using-clp/ https://www.uber.com/en-CA/blog/modernizing-logging-with-clp-ii/ We'd be happy to discuss collaboration opportunities and explore how CLP can address your specific use case. Best regards — Reply to this email directly, view it on GitHub <#1135 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZHZILH6DWLGVSG3Q2LT3UFVODAVCNFSM6AAAAACCTZU3OSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINBYHA2TEMQ>. You are receiving this because you were mentioned.

[hoophalab]

Our newest benchmark is located here: https://benchmarks.yscope.com/log-archival-bench/. We also have a blog explaining the methodology of the benchmark: https://blog.yscope.com/log-archival-bench-a-framework-for-comparing-log-archival-analytics-tools-160e98b9d189

Unfortunately, we didn't compare ourselves with Splunk in the latest benchmark. However, we did a comparison with Splunk before.

Workload:
The unstructured log dataset, hadoop-258GB, contains log events generated by Hadoop when running workloads from the HiBench Benchmark Suite on three Hadoop clusters, each containing 48 data nodes. The dataset is a 258 GiB subset of a larger 14 TiB dataset (specifically, workers 1-3).

Ingestion:

	CLP	Splunk
Compression Ratio	73.6 : 1 (Compressed Size 3.5 GB)	2.9 : 1 (Compressed Size 88.7 GB)
Ingestion Mem Usage	1.38 GB	0.65 GB
Ingestion Speed	83.33 MB/s	7.71 MB/s

Search:

	CLP	Splunk
Memory Usage	1.07 GB	0.69 GB
Cold Run Avg Latency (13 Queries)	11.68 s	146.66 s
Hot Run Avg Latency (13 Queries)	11.19 s	28.01 s

[hoophalab]

We've also applied our technology to other companies. For example, this blog discusses how an anonymous Tier-1 electric vehicle manufacturer leveraged CLP to deploy distributed log analytics at the edge:
https://blog.yscope.com/clp-on-edge-in-vehicle-log-compression-and-analytics-on-over-a-million-vehicles-8e4dd9f70a07

You are also welcome to discover more improvements on our webpage and Medium blogs:
Webpage: https://yscope.com/
Blogs: https://blog.yscope.com/

We would also be happy to have an offline discussion with you to see if our technology fits your needs.

[MyraBaba]

Hi, One more question . Is CLP multithreaded ? Our searches always using 1 core an not utilizing all cpu power ? Best

…

On 23 Sep 2025, at 23:33, hoophalab ***@***.***> wrote: We've also applied our technology to other companies. For example, this blog discusses how an anonymous Tier-1 electric vehicle manufacturer leveraged CLP to deploy distributed log analytics at the edge: https://blog.yscope.com/clp-on-edge-in-vehicle-log-compression-and-analytics-on-over-a-million-vehicles-8e4dd9f70a07 You are also welcome to discover more improvements on our webpage and Medium blogs: Webpage: https://yscope.com/ Blogs: https://blog.yscope.com/ We would also be happy to have an offline discussion with you to see if our technology fits your needs. — Reply to this email directly, view it on GitHub <#1135 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZHYVF3FXENA2TV7CHAT3UGVB5AVCNFSM6AAAAACCTZU3OSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINBZGA3DKMA>. You are receiving this because you were mentioned.

[hoophalab]

Yes, CLP parallelizes compression and search at the granularity of individual archives.

CLP is designed for scalable operation, with each archive acting as a fully independent unit—making archive-level granularity the foundation of our scaling strategy. Users frequently query massive log volumes, ranging from hundreds to hundreds of thousands of archives. This architecture also underpins our multi-user support. For single-archive queries, especially against remote cold storage like S3, we aim for response times in the range of seconds to under a minute. Single-threaded performance is more than sufficient for these workloads.