Author Archives: dniswhite

Development of a Swift Vapor App using WebSockets

I have been working with WebSockets since their release. While I like the idea of a browser based full duplex communication between the client and server one of my favorite interests with WebSockets has been getting multiple browser based apps to communicate with each other in realtime.

In this article I will give an example for a realtime application where users can collaborate on a shared html5 canvas using a jQuery plugin I wrote a few years back. I will be using Swift on the server side with the web framework Vapor.

For more information about the plugin that I developed you can review an article that I wrote for it here.

Requirements

You will of course need to have Swift 3.x and Vapor installed on your local machine and if you haven’t already done this there is a great step by step tutorial you can follow here.

Don’t stop here If you think Swift is only for Apple OS’s because it can actually be run on multiple OS’s and the above instructions will given you an example of setting it up to run on Ubuntu. I wrote this article demonstrating how to setup a Virtual Ubuntu machine for Vapor.

Finally the rest of this post assumes you already have a basic understanding of HTML, Javascript, Swift and Vapor. For those wanting to learn more about Vapor there are some great how to videos and articles to get started here.

Setting up Client

I don’t plan to cover the details of laying out the page but for a quick demonstration of what everything will look you can go here.

Talking with Server

To make communications between the clients simple I will be using JSON objects.

The following code will be used to store and manage the WebSocket communications that are sent and received:

function dooScribConnection(host) {
    var scrib = this;
    scrib.ws = new WebSocket('wss://' + host);
            
    scrib.ws.onopen = function() {
        var connRequest = JSON.stringify({'command': 'connect', 'username': id})
        scrib.ws.send(connRequest)
    };
            
    scrib.ws.onmessage = function(event) {
        var msg = JSON.parse(event.data);
                
        if (msg.command == "click") {
            prevPoint[msg.username] = msg;
        } else if (msg.command == "paint") {
            surface.drawLine(prevPoint[msg.username].X, prevPoint[msg.username].Y, msg.X, msg.Y, msg.color, msg.pen);
            prevPoint[msg.username] = msg;
        } else if (msg.command == "clear") {
            surface.clearSurface();
        }
    };
            
    scrib.send = function(event) {
        scrib.ws.send(event);
    };
};

The connection to the server is established as soon as the WebSocket object is created. The onOpen function is called once that connection is established. By making a call to send information to the server here it will allow the server to store instance information about each of its connections. To make sure that each user is uniquely identified the id var is established using the following:

var id = Math.round($.now()*Math.random());

For messaging to the server I added the send function which will send the JSON objects that are passed as a parameter to it.

Whenever a message is received the onmessage function is called and from there you can switch on the command type (added to the JSON message) and take the appropriate action. Currently the following commands (JSON objects) are of interest for handling inside the browser:

  • click – this is used to establish the starting point for when the user has started drawing.
  • paint – this is used to draw a line from the previous point to the current one.
  • clear – a command that allows users to clear the drawing canvas.

Setting up Canvas (DooScrib plugin)

The following code is for setting up the jquery plugin as soon as the page has been loaded and is ready:

$(document).ready(function(){
    var height = $('#raw() { #surface }').height();
    var width = $('#raw() { #surface }').width();

    dooscrib = new dooScribConnection(window.location.host + "/dooscrib");
    surface = new $('#raw() { #surface }').dooScribPlugin({
        width:width,
        height:400,
        cssClass:'pad',
        penSize:4,
                                                                  
        onMove:function(e) {
            var msg = JSON.stringify({'command': 'mousemove', 'username': id, 'X': e.X, 'Y': e.Y });
            dooscrib.send(msg);
        },
                                                                  
        onClick:function(e) {
            var msg = JSON.stringify({'command': 'click', 'username': id, 'X': e.X, 'Y': e.Y});
            dooscrib.send(msg);
        },
                                                                  
        onPaint:function(e) {
            var msg = JSON.stringify({'command': 'paint', 'username': id, 'X': e.X, 'Y': e.Y,'pen': surface.penSize(),'color':surface.lineColor()});
            dooscrib.send(msg);
        },
                                                                  
        onRelease:function(e) {
            var msg = JSON.stringify({'command': 'release', 'username': id, 'X': e.X, 'Y': e.Y});
            dooscrib.send(msg);
        }
    });
});

The drawing for the current surface is handled by the plugin which then messages everything that it is being done. With each message there is JSON message created which is then sent to the server which will redirect to all the other users.

Setting up Server

For the server implementation I created the following Controller object which handles all of the WebSockets communications as well as routes for the different web pages:

final class ScribController {
    var dooscribs : [String: WebSocket]
    var droplet : Droplet

    init(drop: Droplet) {
        dooscribs = [:]
        droplet = drop

        droplet.get("", handler: scribRequest)
        droplet.get("about", handler: aboutRequest)
        droplet.socket("dooscrib", handler: socketHandler )
    }
    
    func aboutRequest(request: Request) throws -> ResponseRepresentable {
        return try droplet.view.make("about")
    }

    func scribRequest(request: Request) throws -> ResponseRepresentable {
        return try droplet.view.make("socket")
    }
    
    func socketHandler(request: Request, socket: WebSocket) throws {
        var scribUser: String? = nil
        
        // create an active ping to keep connection open
        try background {
            while socket.state == .open {
                try? socket.ping()
                drop.console.wait(seconds: 5)
            }
        }
        
        socket.onText = { socket, message in
            let json = try JSON(bytes: Array(message.utf8))
            
            guard let msgType = json.object?["command"]?.string, let user = json.object?["username"]?.string else {
                return
            }
            
            if msgType.equals(any: "connect") {
                scribUser = user
                self.dooscribs[user] = socket
                
                let response = try JSON(node: [
                    "command":"connected",
                    "username": user
                    ])
                
                // send a connect response to everyone including self
                for (_, connection) in self.dooscribs {
                    try connection.send(response)
                }
            } else if (msgType.equals(any: "clear")) {
                for (_, connection) in self.dooscribs {
                    try connection.send(json)
                }
            } else {
                // send message to everyone (minus self)
                for (scrib, connection) in self.dooscribs {
                    if (!scrib.equals(any: user)) {
                        try connection.send(json)
                    }
                }
            }
        }
        
        socket.onClose = { ws, _, _, _ in
            guard let user = scribUser else {
                return
            }
            
            let disconn = try JSON(node: [
                "command": "disconnect",
                "username": user
                ])
            
            // tell everyone (minus self) about disconnect
            for (remote, connection) in self.dooscribs {
                if (!remote.equals(any: user)) {
                    try connection.send(disconn)
                }
            }
            
            self.dooscribs.removeValue(forKey: user)
        }
    }
}

I won’t be covering the get route handlers that are established in the controller with any great detail. From the code you however you can see that they are sending back the pages the user requested.

Sending JSON Messages

Much to my surprise the WebSocket class does not have a function for sending JSON packets so I created the following extension to handle that:

extension WebSocket {
    func send(_ json: JSON) throws {
        let data = try json.makeBytes()
        
        try send(data.string)
    }
}

Handling Messages

Messages are handled via the onText EventHandler which is called with a WebSocket and String passed in via closure. The string data is parsed into a JSON object so that command and user id that should be sent with each message.

For the connect message the user id and accompanying WebSocket are saved in the dooscribs dictionary and then a connected message is created and, much like the clear message, it gets to sent to all of the connections.

The other messages that are received get forwarded to all of the current connections with an exception for the one where the message.

Handling Connections and Users

Each connection will be stored in the dooscribs dictionary which is a pairing of the unique id that was generated by the browser and the WebSocket for their connection.

Whenever a connection is closed (user leaves page, closes browser) the onClose handler is called informing of the event. For cleanup purposes send a disconnect message to all of the clients that are stored in the dooScribs dictionary, minus the connection that just closed, as well as remove that user from the dictionary.

Issues – Random Disconnection

Something I noticed in my development was that clients were disconnecting for what seemed at the time as unknown reasons. After some research I found that “quiet” connections will automatically get disconnected. To handle this I created the background handler so that it would ping the socket every 5 seconds to keep channels open.

Conclusion

Hopefully this gives you some ideas for processing realtime data in your applications. In the future I plan to expand the dooScrib application so that it can handle different rooms as well text based messaging. The code for the dooScrib plugin as well a Node.js Implementation and this Vapor implementation can be founder here.

For now enjoy and as always Happy Coding!!!

Resolutions and Challenges

Well an old year has gone and new one is on its way.

Achievements of 2016

Each year, like everyone else, I make some resolutions and do my best to achieve each of them during the year. I can say that last years achievement I am most proud of was giving up soda and any other carbonated drinks. After a lifetime of drinking carbonated drinks this was harder to achieve than I expected it would be.

I can now say though that while eating at restaurants I really enjoy drinking water now and when I see my savings on the check it makes me feel a bit better.

At the same time I wanted to make sure that I didn’t replace carbonated drinks with high sugar drinks. Again a success and I found a new taste for a Green Tea.  Success!!

On the technical front last year was quiet and I decided to focus my time on getting more comfortable with my current knowledge set and just enjoy what I had.

Resolution for 2017

For my personal achievements I want to stop going out to eat as much as we normally do and instead eat at home more often. When I discussed this with my wife it caused some initial frustrations but I explained to her with the savings it will help achieve my other resolution for the next year. I would like for our family to take either a European vacation or something that allows us to travel to the east coast.

On the technical front I have the following planned:

  • Rewrite dooscrib.com with a Swift and Vapor backend.
  • Complete the first phase of smart mirror project.
  • Update iOS apps with more features.
  • Start going to more programmer meetings again.
  • Do a presentation on Vapor and Server Side Swift.

As I complete or work on some of my resolutions for the new year I will write about them and share my experiences.

Until then Happy Coding!!!

 

VNC Experiences on Raspberry Pi

I wanted to share some experiences of getting VNC setup on my Raspberry Pi.

How hard could the experience be? A basic search on google and there are a ton of tutorials and examples to follow getting everything setup. Being a fan of adafruit I followed a detailed tutorial they had for setting up VNC which can be found here.

Initially it was working great till I noticed that I wasn’t accessing the current session. For most configurations I suspect this works fine but for my current project I want my VNC session to use the active session that I would see if I had a KVM (keyboard, video, mouse) plugged into my Pi.

Certainly I am not the first to notice or expect this. After trying different configurations and researching things further I discovered that the VNC tool used in all of the tutorials (tightvnc) doesn’t support access to the current session. Apparently in order to achieve this I would have to install and run X11VNC.

The following is the steps that I took to get everything setup:

Install X11VNC

Open a terminal session on your Raspberry Pi and run the following:

sudo apt-get update
sudo apt-get install X11VNC

When installing X11VNC it will want you to confirm the install so make sure to answer Yes to these questions. After it’s done installing you could run x11vnc but you’ll probably want it to work automatically after restarting your Pi.

Setting up X11VNC

In your terminal session you will want to setup a password that can be used when connecting. To set this up you will want to do the following:

x11vnc --storepasswd

When you are done x11vnc should report back that the password was written at /home/pi/.vnc/passwd. For some assurances I checked the file because I didn’t like the idea of password being stored in plain text.

Next you’ll want to set everything up so that x11vnc is running whenever your pi is started. In your terminal session change to the config directory.

cd /home/pi/.config

There should already be an autostart directory but go ahead and run a list to see everything that exists in the directory.

ls

If you do not have an autostart directory then you will need to create one by using mkdir.

Go ahead and change to the autostart directory.

cd autostart

Next you’ll want to create an entry for x11vnc.

nano X11VNC.desktop

Add the following changes to the file:

[Desktop Entry]
Encoding=UTF-8
Type=Application
Name=X11VNC
Exec=x11vnc -forever -usepw -httpport 5900
StartupNotify=false
Terminal=false
Hidden=false

To close hit Ctrl-X to keep the changes and exit. Next you will want to restart your Pi.

Testing X11VNC

Your configuration might be different than mine however I am running OS X and I can use the built in VNC viewer from Finder. By clicking on the desktop or if you already have a Finder window open you can do a Cmd-K to connect to a VNC Server.

You will want to enter the address of your Pi using the following format:

vnc://<IP Address>:<Port Number>

If you need to get the IP address of your Pi you can run ifconfig from a terminal session on it. Once you have entered the Server Address Click on Connect and you should be prompted to enter the password that you supplied.

You should now be connected.

Enjoy and as always Happy Coding!!!