Get started
This guide will help you get started with RisingWave. We will cover:
Install and run RisingWave
You can run RisingWave in these ways:
- Use the pre-built library (Linux)
- Install and run from a Docker image (Linux & macOS)
- Set up a multi-node cluster via Docker (Linux & macOS)
- Build from the source code (Linux & macOS)
Use the pre-built library (Linux)
Download the pre-built library.
wget https://github.com/singularity-data/risingwave/releases/download/v0.1.11/risingwave-v0.1.11-x86_64-unknown-linux.tar.gz
Unzip the library.
tar xvf risingwave-v0.1.11-x86_64-unknown-linux.tar.gz
Run RisingWave.
./risingwave playground
RisingWave is now started.
Install and run from a Docker image (Linux & macOS)
You can install and run RisingWave from a Docker image on x86_64 systems. Images for ARM64 systems (including macOS devices with an Apple M1 chip) might be available for testing purpose, but it is not guaranteed.
As prerequisites, you need to install Docker Desktop in your environment.
Start RisingWave in single-binary playground mode:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 ghcr.io/singularity-data/risingwave:v0.1.11 playground
Set up a multi-node cluster via Docker (Linux & macOS)
You can set up a full-feathered RisingWave cluster via Docker Desktop. The cluster is composed of multiple RisingWave components, including:
- A frontend node
- A compute node
- A meta node
- A compactor node
RisingWave also incorporates these third-party components:
- Grafana
- Etcd
- MinIO
- Prometheus
Therefore, it will start 8 processes.
As prerequisites, you need to install Docker Desktop in your environment. Ensure that it is running before launching the cluster.
Then, clone the risingwave-demo repository.
git clone https://github.com/singularity-data/risingwave-demo.git
Now navigate to the docker
directory and start the cluster from the docker-compose file.
cd docker
docker-compose up -d
Build from source (Linux & macOS)
You can build from source on both x86_64 and ARM64 systems (including macOS devices with an Apple M1 chip).
clone the risingwave repository.
git clone https://github.com/singularity-data/risingwave.git
Install dependencies.
RisingWave has the following dependencies. Please ensure all the dependencies have been installed before running RisingWave.
- Rust
- CMake
- Protocol Buffers
- OpenSSL
- PostgreSQL terminal (14.1 or higher)
- Tmux
Select your operating system and run the following commands to install the dependencies.
- macOS
- Linux
brew install postgresql cmake protobuf openssl tmux
Run one of the following cammands to install rustup:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
or
brew install rustup-init && rustup-init
sudo apt update
sudo apt upgrade
sudo apt install make build-essential cmake protobuf-compiler curl openssl libssl-dev libcurl4-openssl-dev pkg-config postgresql-client tmux lld
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Run RisingWave.
To run RisingWave, in the terminal, navigate to the directory where RisingWave is downloaded, and run the following command.
./risedev playground
All services in RisingWave will be started.
Connect to RisingWave
After RisingWave is started, you can connect to it via the Postgres interactive terminal psql
.
psql -h localhost -p 4566 -d dev -U root
You can now issue SQL queries to manage your streams and data.
Connect to a streaming source
Use the CREATE SOURCE
command to connect RisingWave to a streaming source.
To connect to a Kafka topic:
CREATE SOURCE KAFKA_TOPIC_1 (
COLUMN_NAME DATA_TYPE, ...
)
with (
connector='kafka',
kafka.topic='demo_topic',
kafka.brokers='172.10.1.1:9090,172.10.1.2:9090',
kafka.scan.startup.mode='earliest|latest',
kafka.time.offset='140000000',
kafka.consumer.group='XXX_CONSUMER_NAME'
)
ROW FORMAT 'json'
[ROW SCHEMA LOCATION 's3://path'];
For supported streaming sources and sample SQL statements, see CREATE SOURCE.
Query and manage data
RisingWave uses Postgres-compatible SQL as the interface to manage and query data.
Before we start, ensure that you have connected to RisingWave via psql
.
Now let us create a table to store data about taxi trips.
CREATE TABLE taxi_trips(
id VARCHAR,
distance DOUBLE PRECISION,
duration DOUBLE PRECISION
);
We want to create a materialized view to dynamically calculate the average speed of all trips.
CREATE MATERIALIZED VIEW mv_avg_speed
AS
SELECT COUNT(id) as no_of_trips,
SUM(distance) as total_distance,
SUM(duration) as total_duration,
SUM(distance) / SUM(duration) as avg_speed
FROM taxi_trips;
Now let us add some data to the table.
INSERT INTO taxi_trips
VALUES
('1', 4, 10);
We can now query the average speed.
SELECT * FROM mv_avg_speed;
Here is the result we get.
no_of_trips | total_distance | total_duration | avg_speed
-------------+----------------+----------------+------------
1 | 4 | 10 | 0.4
Now let us add a new record.
INSERT INTO taxi_trips
VALUES
('2', 6, 10);
As soon as we insert the new record, the materialized view mv_avg_speed
will be refreshed to re-calculate the results. Let us see if the results are updated.
SELECT * FROM mv_avg_speed;
Here is the result we get.
no_of_trips | total_distance | total_duration | avg_speed
-------------+----------------+----------------+------------
2 | 10 | 20 | 0.5
You can see that the results are based on the two rows of data. The calculation is performed automatically behind the scene. No matter how many more rows of data we insert, we can always get the latest results by querying the values of the materialized view.
Creating a materialized view from a source is similar.
CREATE MATERIALIZED VIEW debezium_json_mysql_mv
AS
SELECT COLUMN1, COLUMN2, COLUMN3 FROM debezium_json_mysql_source;
With RisingWave, you can also create a materialized view from an existing materialized view.
CREATE MATERIALIZED VIEW m3
AS
SELECT m1.v1 as m1v1, m1.v2 as m1v2, m2.v1, m2.v2
FROM m1
INNER JOIN m2 ON m1.v1 = m2.v1;
For the complete list of supported SQL commands, please navigate to SQL reference → Commands.