Update (2018-12-8): I completely forgot to point out in the original post that RSPM can be used to publish rpackages directly from a git repo. This is a brilliant solution for continuous deployment type scenarios. Check the docs for more information!
If you’ve read any of my previous posts you’ll know I’m a big fan of RStudio Package Manager (RSPM). It’s a great tool for enterprise users of R to take control of the way they distribute R packages within their organisation.
I’ve been working on a large corporate install of RStudio Connect and Package Manager recently, but I ran into a problem when writing the documentation that I just couldn’t think my way around, until now.
RSPM is controlled by a nifty command line tool called, unsurprisingly, ‘rspm’. The rspm utility let’s you create your initial repo config as well as add and remove packages from your configurations once they’re in use.
This is a fantastic way to manage your repos and packages and works really well when implementing CI/CD solutions for package build and publication. When publishing a package manually though, it can be a little problematic, depending on your existing knowledge of Linux and how locked down your local IT environment is.
The basic steps to publish a package manually are as follows:
- Build a source package from your code
- Transfer the source package to your RSPM server
- Open your transfer software (WinSCP (Windows) and Filezilla (x-platform) are popular options)
- Connect to the RSPM server
- Transfer the package file
- Disconnect and close the application
- Use the RSPM cli to add the package to your repo
- Open your SSH software (PuTTY is a de facto standard on Windows, other platform have SSH built in)
- Connect to the RSPM server
- Run the command to add the newly transferred source package
- Disconnect and close the SSH client
None of this is an issue in and of itself. These are all fairly standard things that people working with Linux servers every day do all the time.
But a lot of people don’t work with Linux every day and having transfer software and an SSH client in a Windows environment is the exception, rather than the norm.
I know that the client that got me thinking about this in the first place doesn’t have PuTTY or WinSCP. He can get them, they’re available to users on request in this particular environment, but it’s still not ideal. Requesting software and getting approval can take time and then learning to use it properly takes time too.
Anyway, I wrote up the process using the WinSCP-and-PuTTY method, but it’s been bugging me that there isn’t something more appropriate for this client ever since. After the kids had gone to bed and with a glass of wine by my side, I still couldn’t help thinking about the process. And then the penny finally dropped.
I remembered that I’d read an annoucement from Jeroen Ooms/ROpenSci about a package for working with SSH directly from R. I had installed it at the time and had a quick play around with it, but it suddenly seemed like I had a good use for it. Anyway, one quick skim of the docs later and we can now publish new packages to RSPM directly from R, without needing any external tools at all.
> install.packages("ssh")
Create a new SSH session to the RSPM server:
> session <- ssh::ssh_connect("username@rspm-server")
This will pop up a password box (you can configure it to use SSH keys if you’d prefer). Enter the password for the specified account.
Then upload the source package using the session we just created:
> ssh::scp_upload(session, "package_ver.tar.gz")
[100%] C:\Users\username\Documents\package_ver.tar.gz
[1] "./pkgs/"
Now add the package to the internal repo, which here is also called ‘internal’:
> ssh::ssh_exec_wait(session, "rspm add --source=internal --path=./package_ver.tar.gz")
Added package './package_ver.tar.gz'.
RSPM takes care of adding the package, and archiving any older versions should one exist.
Finally, close the session.
> ssh::ssh_disconnect(session)
You should now see your new package version in the RSPM web-UI.
Not only is it possible to publish packages to RSPM without using anything other than R, but it can be done in only 4 function calls. This is going to be great news for my client, but it does mean I’ve got to re-write that documentation on Monday morning!