Skip to content

Oxidise#51

Open
rerpha wants to merge 2 commits intomainfrom
oxidise
Open

Oxidise#51
rerpha wants to merge 2 commits intomainfrom
oxidise

Conversation

@rerpha
Copy link
Copy Markdown
Collaborator

@rerpha rerpha commented Mar 24, 2026

Convert saluki to rust.

Changes to note:

  • listen and consume have been merged, as they were quite similar anyway. README and --help should clarify how to use consume (which is aliased to listen for muscle memory's sake!)
  • play has not been carried over as I ran out of time to be able to call this personal development. rewrite play in rust #50 exists to add it when we need to use it again.
  • howl is a bit faster than python, though I haven't tried the "cheat" way of only constructing one set of det values but sending them x times a frame.
  • have added a dockerfile so you don't actually need rust installed to use saluki.
  • I haven't gone mad with unit tests as again I've run out of time (I know I know TDD and so on...)

also note that the CI is currently failing at the time of writing but only because one of the apt mirrors seems to be 404ing? working now

I may also need a hand benchmarking howl plz @Tom-Willemsen

@rerpha rerpha marked this pull request as ready for review March 25, 2026 15:04
@rerpha rerpha requested a review from Tom-Willemsen March 25, 2026 15:04
Copy link
Copy Markdown
Member

@Tom-Willemsen Tom-Willemsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give this a more thorough functional test tomorrow...

Comment on lines +23 to +25
- name: install curl-dev
if: ${{ matrix.os == 'ubuntu-latest' }}
run: sudo apt-get install -y libcurl4-openssl-dev
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not saying this is wrong, but an alternative that avoids system dependencies is:

[target.'cfg(windows)'.dependencies]
rdkafka = { version="0.39.0", features = ["cmake_build"] }

[target.'cfg(unix)'.dependencies]
rdkafka = { version="0.39.0", features = ["cmake_build", "curl", "ssl-vendored"] }

}
}
counter += 1;
if (num_messages.is_some() && counter == num_messages.unwrap())
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (num_messages.is_some() && counter == num_messages.unwrap())
if Some(counter) == num_messages

and similar elsewhere?


let start_time = SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just generally: please try to avoid bare unwrap() - it gives almost no context.

Prefer .expect("some reason why this can never fail") for truly infallible operations, or Result for anything that can legitimately fail.

fbb.finished_data()
}

#[allow(clippy::too_many_arguments)]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is pretty much exactly how I'd written it in the python too, with a zillion arguments, but I think the more extensible approach is to pull all of the "howl config" arguments into a HowlConfig struct, instantiated once in main, and passed through as &HowlConfig to anywhere that needs it.

This avoids multiple methods all taking a similar huge list of arguments.

Comment on lines +128 to +129
mut current_job_id: String,
) -> String {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you take a mutable reference you could avoid needing to return the new String here.

det_max: i32,
mut current_job_id: String,
) -> String {
// get currnet time
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// get currnet time
// get current time


for _ in 0..messages_per_frame {
match producer.send(
BaseRecord::to(format!("{topic_prefix}_rawEvents").as_str())
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does just:

Suggested change
BaseRecord::to(format!("{topic_prefix}_rawEvents").as_str())
BaseRecord::to(&format!("{topic_prefix}_rawEvents"))

work?

println!("Each ev44 is {ev44_size} bytes");

let producer: ThreadedProducer<DefaultProducerContext> = ClientConfig::new()
.set("bootstrap.servers", broker)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How fast were you able to howl with just this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants