Integration with sidekiq-status

[riyaadh-abrahams]

Hi

I was looking for a good way to integrate with sidekiq-status, and came up with a base job class that will add the various statuses. Any feedback would be helpful, but maybe there could be a way to get something more integrated? I am more that willing to help with some guidance as I am sure this is not the best way

class BaseJob < Gush::Job
  include Sidekiq::Status::Worker

  def perform(*args)
    setup_initial_metadata(args)
    store_for_id id, initial_metadata, @expiration

    begin
      store_status id, :working, @expiration
      execute(*args)
      store_status id, :complete, @expiration
    rescue Sidekiq::Status::Worker::Stopped
      store_status id, :stopped, @expiration
    rescue SystemExit, Interrupt
      store_status id, :interrupted, @expiration
      raise
    rescue Exception
      store_status id, :failed, @expiration
      raise
    end
  end

  def store(data, expiry = @expiration)
    data.each do |key, value|
      store_for_id(id, { key => value }, expiry)
    end
  end

  private

  def setup_initial_metadata(args)
    @initial_metadata = {
      jid: id,
      status: :queued,
      worker: klass.to_s,
      workflow_data: as_json,
      args: args
    }
  end

  def execute(*args)
    # Override this method in your job class with the specific job logic
    raise NotImplementedError, "You must implement execute in your job class"
  end

  def initial_metadata
    @initial_metadata
  end

end

Then it can be used like this

  class JobOneJob < BaseJob

    def execute(*args)
      puts 'test'
      store message: 'wow'

    end
  end

If you set up sidekiq-status web ui, you can see the results like this

[natemontgomery]

I think the idea has some great merits, definitely a Web UI for status would be handy in many cases. I don't know that using the Sidekiq UI to get there is going to give me all of things I would want, but that is me. It could be a starting point, but you also will be taking on a commitment to map to the Sidekiq data structures (obvious I know). I am not sure, but it seems like that could over time introduce odd integration problems, the kind I'm not sure the maintainers of Gush would want to solve for.

Personally, I would like to have whatever UI was designed or gem used provide for awareness of the workflows and not just the individual jobs. Visualizing the workflow runs is a win for me and I have done some rake tooling in a couple cases to introduce a 'tree-like' CLI output for workflows composed from various job states that I managed in my own models. This type of solution felt good enough, but a Web UI would be much better for sure!

Of course, not being a maintainer I am not sure if this fits more with their vision than I am imagining, since you can use Sidekiq for the backend it seems worth consideration. To try and be more immediately helpful though, I do notice that your Time Elapsed fields all appear to be negative, is that something you have already debugged or am I just misunderstanding your pic there?

[pokonski]

Using Sidekiq UI has one major issue: not everyone with ActiveJob uses Sidekiq as its backend. In Postgres projects I use GoodJob these days.

Granted, that's probably a minority 😅

With recent improvements to pagination and listing all workflows, maybe it's time to bring UI for Gush back from the dead.

[natemontgomery]

A Gush UI sounds like a great project :) I would be happy to take a stab at it. I imagine we would be ok with using a Hotwire/Stimulus based approach? I have been thinking that would be something nice to have for a while now. If someone already has it in their plans, I would be glad to help out.

[pokonski]

Absolutely okay with Hotwire/Stimulus :)