Get started

This guide will help you get started with RisingWave. We will cover:

• Install and run RisingWave
• Connect to RisingWave
• Connect to a streaming source
• Query and manage data

Install and run RisingWave

You can run RisingWave in these ways:

• Use the pre-built library (Linux)
• Install and run from a Docker image (Linux & macOS)
• Set up a multi-node cluster via Docker (Linux & macOS)
• Build from the source code (Linux & macOS)

Use the pre-built library (Linux)

1. Download the pre-built library.

   wget https://github.com/singularity-data/risingwave/releases/download/v0.1.11/risingwave-v0.1.11-x86_64-unknown-linux.tar.gz

2. Unzip the library.

   tar xvf risingwave-v0.1.11-x86_64-unknown-linux.tar.gz

3. Run RisingWave.

   ./risingwave playground

   RisingWave is now started.

Install and run from a Docker image (Linux & macOS)

You can install and run RisingWave from a Docker image on x86_64 systems. Images for ARM64 systems (including macOS devices with an Apple M1 chip) might be available for testing purpose, but it is not guaranteed.

As prerequisites, you need to install Docker Desktop in your environment.

Start RisingWave in single-binary playground mode:

docker run -it --pull=always -p 4566:4566 -p 5691:5691 ghcr.io/singularity-data/risingwave:v0.1.11 playground

Set up a multi-node cluster via Docker (Linux & macOS)

You can set up a full-feathered RisingWave cluster via Docker Desktop. The cluster is composed of multiple RisingWave components, including:

• A frontend node
• A compute node
• A meta node
• A compactor node

RisingWave also incorporates these third-party components:

• Grafana
• Etcd
• MinIO
• Prometheus

Therefore, it will start 8 processes.

As prerequisites, you need to install Docker Desktop in your environment. Ensure that it is running before launching the cluster.

Then, clone the risingwave-demo repository.

git clone https://github.com/singularity-data/risingwave-demo.git

Now navigate to the docker directory and start the cluster from the docker-compose file.

cd docker
docker-compose up -d

Build from source (Linux & macOS)

You can build from source on both x86_64 and ARM64 systems (including macOS devices with an Apple M1 chip).

1. clone the risingwave repository.

   git clone https://github.com/singularity-data/risingwave.git

2. Install dependencies.

   RisingWave has the following dependencies. Please ensure all the dependencies have been installed before running RisingWave.

   • Rust
   • CMake
   • Protocol Buffers
   • OpenSSL
   • PostgreSQL terminal (14.1 or higher)
   • Tmux

   Select your operating system and run the following commands to install the dependencies.

macOS

brew install postgresql cmake protobuf openssl tmux

Run one of the following cammands to install rustup:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

or

brew install rustup-init && rustup-init

Linux

sudo apt update
sudo apt upgrade
sudo apt install make build-essential cmake protobuf-compiler curl openssl libssl-dev libcurl4-openssl-dev pkg-config postgresql-client tmux lld
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

3. Run RisingWave.

   To run RisingWave, in the terminal, navigate to the directory where RisingWave is downloaded, and run the following command.

   ./risedev playground

   All services in RisingWave will be started.

Connect to RisingWave

After RisingWave is started, you can connect to it via the Postgres interactive terminal psql.

psql -h localhost -p 4566 -d dev -U root

You can now issue SQL queries to manage your streams and data.

Connect to a streaming source

Use the CREATE SOURCE command to connect RisingWave to a streaming source.

To connect to a Kafka topic:

CREATE SOURCE KAFKA_TOPIC_1 (
   COLUMN_NAME DATA_TYPE, ...
)
with (
   connector='kafka',
   kafka.topic='demo_topic',
   kafka.brokers='172.10.1.1:9090,172.10.1.2:9090',
   kafka.scan.startup.mode='earliest|latest',
   kafka.time.offset='140000000',
   kafka.consumer.group='XXX_CONSUMER_NAME'
)
ROW FORMAT 'json' 
[ROW SCHEMA LOCATION 's3://path'];

For supported streaming sources and sample SQL statements, see CREATE SOURCE.

Query and manage data

RisingWave uses Postgres-compatible SQL as the interface to manage and query data.

Before we start, ensure that you have connected to RisingWave via psql.

Now let us create a table to store data about taxi trips.

CREATE TABLE taxi_trips(
    id VARCHAR,
    distance DOUBLE PRECISION,
    duration DOUBLE PRECISION
);

We want to create a materialized view to dynamically calculate the average speed of all trips.

CREATE MATERIALIZED VIEW mv_avg_speed
AS
    SELECT COUNT(id) as no_of_trips,
    SUM(distance) as total_distance,
    SUM(duration) as total_duration,
    SUM(distance) / SUM(duration) as avg_speed
    FROM taxi_trips;

Now let us add some data to the table.

INSERT INTO taxi_trips
VALUES
    ('1', 4, 10);

We can now query the average speed.

SELECT * FROM mv_avg_speed;

Here is the result we get.

 no_of_trips | total_distance | total_duration | avg_speed      
-------------+----------------+----------------+------------
           1 |              4 |             10 | 0.4

Now let us add a new record.

INSERT INTO taxi_trips
VALUES
    ('2', 6, 10);

As soon as we insert the new record, the materialized view mv_avg_speed will be refreshed to re-calculate the results. Let us see if the results are updated.

SELECT * FROM mv_avg_speed;

Here is the result we get.

 no_of_trips | total_distance | total_duration | avg_speed      
-------------+----------------+----------------+------------
           2 |             10 |             20 | 0.5

You can see that the results are based on the two rows of data. The calculation is performed automatically behind the scene. No matter how many more rows of data we insert, we can always get the latest results by querying the values of the materialized view.

Creating a materialized view from a source is similar.

To create a materialized view from a source:

CREATE MATERIALIZED VIEW debezium_json_mysql_mv 
AS 
    SELECT COLUMN1, COLUMN2, COLUMN3 FROM debezium_json_mysql_source;

With RisingWave, you can also create a materialized view from an existing materialized view.

To create a materialized view from existing materialized views:

CREATE MATERIALIZED VIEW m3
AS 
    SELECT m1.v1 as m1v1, m1.v2 as m1v2, m2.v1, m2.v2 
    FROM m1 
    INNER JOIN m2 ON m1.v1 = m2.v1;

For the complete list of supported SQL commands, please navigate to SQL reference → Commands.