Laravel 5: Queues – General

Scheduler: Queues – General


Queues: general info

IMPORTANT:
You have to have queue:listen command running – preferably as daemon – for queues to work.
What is this queue?

Queues allow you to defer the processing of a time consuming task, such as:

  • sending an email,
  • creating report
  • broadcasting Laravel events
  • any other time consuming and clogging normal workflow task

… until a later time.

So, instead of getting user yawning while waiting for some hog-of-a-task to complete, you can remove this task out of workflow and into a queue of similar tasks, that can be done asynchronously later.

 


Jobs

Each of these tasks attached to a queue is called a job.
When you look at them, they are very similar to events/listeners.
They are designed to perform same, or at least similar task.
They even work in a very similar way and both can utilize Queues.

Lets see what’s differs them:

#1.
Job
: helper function dispatch() (job trigger), can call only one job.
Event: helper function event() (event trigger) calls event class, which can call into action more than one listener

job-event
#2
Job Scope

Jobs seem to be created to unload code from Controllers and make them lighter, and seem to be meant for bigger tasks, than Events.

#3
Queue Scope
Events: all you can really do, is that you can enable queuing and that is pretty much it.
Jobs: It is easier to use different connections, like: database, beanstalkd, sqs etc. plus you have bunch of other functionality to mold queue to fit your needs.

 


Connections

Laravel gives us same API, same code, that allows to work with different queues engines, like:

  • synch
  • database
  • redis
  • sqs (Amazon)
  • beanstalkd
Sync:
This one should not be used for anything except for development really, as it works synchronously, so all positives queues bring (async, behind scenes, off workflow) are not utilized here.
Database:

This is a database based queue, so we need a special database table.
We have handy Artisan command that creates migration file for us.

Sometimes you can get errors. In case you get them, just run this:

Now, when queue table migration file is ready, we can push it into a database and create our queue table:

After last command, you should have jobs table in your database.

Redis

Another way to setup your queue is Redis.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.

Redis is much more than cache engine and is great, if you are using multiple servers and have a busy-busy site.
I am not sure, if using Redis makes sense for a single server, slow to medium busy site.
I would use database based queue it this case.

Plus, unified Laravel API allows you to switch painlessly to Redis, if yous site starts to get busy.

More: read here.

SQS

This is a caching service offered by Amazon.
Read here for more.

Beanstalkd

This is a dedicated queuing system.
This is all it does, unlike e.g. Redis, which is much, much more than queue.
More, read here.

 

Connections can be defined in:

There is a default connection:

At the beginning it is set to ‘sync’.

Your system will be using it, when you do not specify queue driver (connection) in code dispatching job to a queue, e.g., when you are not chaining methods like:

  • onQueue(‘some_queue’)
  • onConnection(‘some_connection’)

In addition to default connection, there are other available connections – given they are setup.

You can set something like this:

For above to work, you need to start worker with proper queue order, like:

Standard worker command is being created behind scenes.

If you want to customize your worker, to differentiate between e.g. queue importance, you have to run command yourself.

Of course, you can run such command programmatically from PHP level.

 

Laravel 5: Redis – Setup and Stuff

Redis: setup and stuff


Redis is an open source, advanced key-value store.
It can contain strings, hashes, lists, sets, and sorted sets.
It can be used in Laravel in: queues, cache, broadcasting.
Redis can also be used as a fast, persistent database – great for horizontal projects (many servers), as it can easily replicate on these servers (nodes).

Redis for Windows

Most likely you will need it for local development, as I do.

I develop using XAMPP with PHP 7 on Windows 8.1, so I need Redis for my win-dev environment.
When I am done developing, I will push it onto Linux box, but it does not change the fact, that I need something for Windows now.

Here where Redis for Windows can be found.

NOTE:
For development do not use password, or anything you do using console is going to drive you bananas, as it will scream for that password.

 


Redis for Linux

To get Redis for Linux:

  1. sudo apt-get install redis-tools
  2. sudo apt-get install redis-server
  3. go inside of installation dir and you can use redis-cli command

You can also see here.

 


Using Redis CLI

On Windows, Redis is located int:

So, to access it on console, you have to cd to Redis directory first:

… and then use:

It works same on Linux, just you have to cd to different directory.

Available Redis commands are here.

 


Using redis with Laravel

You have two choices:

  • Predis and
  • PhpRedis

Some links: PhpRedis  (see also here) and Predis.
Both scripts provide the same access wrapper to Redis server.

So, you need to have Redis installed first!

PhpRedis is written directly in C, while Predis is written in PHP, so in some cases Phpredis is faster.

How much faster?
Well Aleksey Korzun took his time to run some benchmarks and here are the results.

 


Redis version

To check Redis version, use this command from within Redis directory in console:

 


To check, if Redis is up, you may use ping

Redis is going to answer with: pong.

 


Setting password

Password can be set in redis.conf file.
In Windows, this file is called: redis.windows.conf.
Password is set around line#443.
Just uncomment this line and add password to it:

Note:
In Windows, any changes made in redis.windows.conf will take effect only after computer is restarted.
So, if you come across same kind of a problem, you need to apply password using console, like this:
Make your password very complex, to avoid being hacked.
Something like: rTY6UNMeyAkjL23YTX*qXG8X1p1u#RDd4v85rQNW@82FM4q should do the trick.
Important:
If out of the blue, you get an error: “failed: NOAUTH Authentication required” and you are on Windows (not tested on Linux/Unix), it may mean that you attempted to set password (requirepass) in file redis.windows.conf (and failed). In such case, after next computer restart, you can get this unexpected error.
Actually, that is why you failed to apply your password in first place, because computer restart is required to get this going.
Solution:
Just comment that out and restart Windows again, or pass your new password, whenever you interact with Redis in your app.
Frankly, when developing, it is good to develop with password in place, as later, it can be tedious to put it in all the right places. Unless you use .env based Redis constant password. Just I am not sure, if you can use it in *.js files, like node scripts. Haven’t tested that yet. Drop me a line, if you did.

 


Protect redis by loopback
Loopback and password
If you protect Redis by loopback, you are taking it effectively from direct access from web.
So, in such case, there is no need for using password.
As long as connecting client is secure and connection is secure (SSL) – in case it is not localhost – Redis is protected.

Protect redis by loopback interface – uncomment (if commented) in redis.conf:

If your redis is not on same server (so 127.0.0.1 cannot be used), explicitly state allowed client:

You can write as many interfaces (IPs) as you need, by listing them with space between IPs, as shown above.

If you are explicitly binding localhost, you may do this including IPV6 notation:

This part:

is IPV 6 equivalent of:

… in IPV4 format.

 


Order of commands in CLI (console) are important!

This will NOT work:

This WILL work:

As you can see, password has to go first and then rest of commands.

If server has no password set, anything passed via ‘-a’ is being disregarded.

 


How to start Redis server.

On Windows, when you install downloaded MSI file (installer), it will start Windows equivalent of daemon process.
Deamon process in Linux is a background, long running process.

But, if for any reason, you stop such process, e.g. by using console command:

… Windows ‘daemon’ is ‘exorcised’ 😉 and thus dead.

To get Redis again, you can:

use console

From within redis folder run command:

note:
Windows do not run daemons, so console will be disabled by process and process will die, when console is closed.

setup Windows task

Start it as Windows task – see here, how it is done.
Example is about running pseudo-chron job based on Windows native tasking system, but this task can be setup in the same way.

One difference:
in ‘New Action’ tab, do not add ‘New Arguments’.

 

Socket.Io – Working Chat and General Stuff

Socket.Io – Working Chat and General Stuff


Socket.io allows for PHP based system to have server pushing data to client without request from client.

Most of that stuff below needs node.js installed on your server.

If you work on Windows just get Windows installer and add node to your Environment Variable path, to make it global.

Similar operation describing registering Composer with EV path is described here.


Socket.Io
Checking version

Same way you can check all stuff installed via np, like: express, ioredis etc.

Installation

Socket.io can be downloaded from here.

It can be installed using node.js npm, like this:

It is good to use ‘–save’ flag to add socket.io to dependencies list in your project package.json file, to keep it up to date with other stuff you are using, or you are about to use.

Usual socket project structure

Socket client

You need a socket.io client, which your project will pull into user’s browser
Something like:

If you do not want to use socket.io CDN (content delivery network) hosted client script, you can download version from here and serve it from your own server, or some other CDN.

Socket server worker

This is usually an index.js file, which is being activated via console command:

This file is a place where you bootstrap and fire all needed elements, like:

  • express framework
  • socket.io and all its channels, connections and broadcasting
  • html/template file (called, when accessing defined port)
  • define a port app is going to be listened on
  • etc.

When bootstrap is neatly tied, you can listen to your app in browser, by listening to defined port like this:

 

Sample Chat

Socket.io home page have chat demo here.

Unfortunately demo does not work.

In downloadable package below, you will find fixed version of that chat.
Of course, full credit goes to creators – I just fixed  few little things.

Edit 2018-03-18: demo works now and they added nice whiteboard.

zip

In archive you will find a drop-in package.
What it means is that all you have to do is:

  1. unzip
  2. drop content of folder /b8 (in archive) to your localhosted location
  3. change ‘b8’ in index.html file to your own localhosted location/domain, e.g. /mychat
  4. you need internet access (dependencies: botstrap, socket client, jquery)
  5. node dependencies: socket.io and express are already in zipped package
  6. open command prompt (windows) or console and go inside of your ‘mychat’ directory (or some other you are using)
  7. type is: node index.js – keep console open
  8. navigate in your browser to: mychat:3000

… and you have working chat open.

Now you can open another window with chat (or two) and see how posted stuff gets passed via sockets between browser windows. Same it would work between people in remote locations.

Here, you can find some nice socket.io cheat-sheet. Thanks Alex.
Just is case, it is also accessible here: socket-cheatsheet

 


Redis

If you wish to store some data, you can use Laravel’s Swiss army knife: Redis (queues, broadcasting, cache, data storage etc.)
More about Redis – here.

 


Redis.Js

There is also Redis API (client) called Redis.Js, that is a great way to work with Redis from within Socket.Io code.
You can find it here.

When installing, it is good to add it as dependency to your package.json like this (note –save):

This will update your package.json.

You need to have Redis installed on your system to use above wrapper (API)!
Other Redis clients

Find the all here.

Also, you may look at ioredis instead of redis.js

 


Express JS Framework

You will need this JS framework to work with sockets.

You can install it like this:

Again, ‘–save’ will add it to package.json as dependency.