Help:Getting Started

From Wikitech

Overview

Cloud Services provides tools, services, and support for technical collaborators who want to contribute to Wikimedia software projects. This page outlines basic first steps to making contributions to this wiki and Wikimedia's technical spaces.

This guide will teach you to:

  • Create a Wikimedia developer account for Toolforge, Cloud VPS, and general use
  • Begin using Gerrit for code review
  • Access an existing project or begin a new project

Step 1: Create a Wikimedia developer account

Most of Wikimedia's technical spaces use a Wikimedia developer account. A developer account is required to:

Toolforge users

Creating an account

  1. Use the Toolforge admin console to create a new developer account.
    • Make careful note of the LDAP username and UNIX shell username you choose
  2. Fill out an access request to join the Toolforge project.
  3. Generate an SSH key that you will use along with your shell account name to log into the Toolforge servers if you do not already have one.
  4. Associate your SSH public key with your Wikitech account using the Preferences > OpenStack tab or Toolforge admin console.
  5. Wait for your access request for the Tools project to be processed (you should receive messages on your Wikitech talk page).

After account creation

  • Use SSH to login to Toolforge. In a command-line terminal, enter: ssh -i ~/.ssh/id_rsa username@login.tools.wmflabs.org (username is the "Instance shell account name" you specified when you created an account. If you are running Windows you will need to use an ssh client like puTTY.
  • Use SSH-based utilities such as scp and sftp to transfer files between Toolforge and your computer.
  • Access MySQL from SSH (e.g. sql enwiki is a shortcut command to connect to the copy of English Wikipedia)
  • Create tools (see Help:Toolforge § Creating a new Tool account).

Common issues

  • Your LDAP username and your shell login username may be different. Visit Settings > Linked accounts in the Toolforge admin console and check your "Shell account name".
  • The passwords you chose for your LDAP account and SSH key may be different.
  • When you login with SSH you are in your personal folder. To quickly go to your tool account enter: become tool_name
  • If you login with WinSCP you will transfer files as yourself, not as your tool. The UNIX group for both accounts is the same so you can make files editable by all maintainers of the tool with:
$ become TOOL_NAME
$ chmod --recursive g+rw .
  • The web service for your tool is not started by default. To start it enter: webservice start
  • When doing file system intensive tasks (like git clone), it may be useful to perform the task on a host-local filesystem (like /tmp) to avoid the speed penalty of NFS. For example, cloning git repositories in /tmp and then moving it to the location you actually need it in can be much faster than cloning it in the original location.
    • Note: the speed penalty can be severe. For example, running wc -l on a 285 MB text file took 0:01 on /tmp and 3:14 over NFS.
  • If you need to use the server for a long time, and want to execute a task even when you're not connected to the ssh, you can use screen. Note that screen needs to be run before become TOOL_NAME. Screen should not be used to run long running scripts. These should be run as grid jobs instead.
  • You might need to use ssh-add on your local computer after creating a new key.

Using multiple ssh agents

If you use multiple ssh-agents (to connect to your personal or company system, for example), see SSH access for more information about setting up a primary and a Cloud VPS agent.

VPS and General Users

Creating an account

Create a Wikimedia developer account on Wikitech using the account creation form.

Uploading an SSH key

Wikimedia Cloud Services products and Wikimedia Code Review use an SSH key to authenticate you. If you already have an SSH key you can use that one, or generate a new one specifically for Wikimedia use; the default public key name generated by ssh-keygen is typically id_rsa.pub.

Upload the generated public key from the 'OpenStack' tab in your Preferences or use the toolsadmin interface.

See Also: SSH Help

You can test your key by attempting to connect to the bastion VPS project: ssh -v bastion.wmflabs.org.

You will only be able to ssh to bastion after you have been added as a member of at least one VPS project (e.g. Toolforge). Users can verify their project membership via OpenStack Browser at https://tools.wmflabs.org/openstack-browser/user/<shellname>

Uploading your key to Wikimedia's Gerrit (Code Review)

Upload your public key to Gerrit (Wikimedia's code review tool).

  1. “Sign In” with your Wikimedia Developer Account
  2. Click “Settings” link in drop down menu under your username at page top right
  3. Click “SSH Public Keys” in left sidebar
  4. Paste your public key into the text field and click “Add” button. Thereafter, the form presents "Add key ..." button instead, for if you have more keys you want to add.

Gerrit/Git committers

If you are just simply getting an account for committing into Gerrit, read some detailed documentation about it

  • mw:Gerrit - mediawiki.org's guide to using Gerrit
  • Help:Git rebase - Because the importance of using fetch and rebase (instead of pull or merge) can't be over-emphasized.

Getting started with Toolforge

Requesting access

Once you have created a developer account, you must request access to the Toolforge project by submitting a Toolforge project membership request.

Requests for access are generally dealt with within days, though response-time may be longer depending on admin availability. If you need immediate assistance, please contact us on the #wikimedia-cloud connect IRC channel.

Receiving access to the 'tools' project

Once your 'tools' project access request has been processed, you will become a member of the Toolforge Cloud VPS project, and will be able to access it using the "UNIX shell username" provided when creating your developer account and the SSH private key matching the public key you supplied for authentication. For more information about accessing the project, please see Using Toolforge and managing your files.

You will be notified on Wikitech that your user rights were changed and that you have been added to the project Nova Resource:Tools. In other words, your Toolforge account is ready for you to use!

Getting started with VPS Projects

Join a Project

In order to work on a Cloud VPS project you need to be granted project membership by a project administrator.

  • Find the project you wish to join using OpenStack browser.
    • To join Toolforge, see the instructions in Getting started with Toolforge
    • If you want to join another project, ask a project administrator for that project. You can find the list of project admins by going to http://tools.wmflabs.org/openstack-browser/project/<project-name>

Add members and admin users to a Project

Request creation of a new Project

If you need to create a new project go to the "Cloud-VPS (Project-requests)" phabricator task and follow the instructions it provides.

Project Instances

Creating Instances

To create instances within a project, see Help:Instances

Access Project Instances

After you have been added to or created a project and successfully added your SSH key to Wikitech and Wikimedia Code Review, read the Help:Access manual.

Using ProxyCommand for SSH

The easiest way to access your project instance is using the ProxyCommand ssh option to directly connect to your instance. 

$ ssh <instance-name>.<project-name>.eqiad.wmflabs

The instance name is given in the "Instance name" column in Special:NovaInstance. Logging in to the instance is just like logging into any computer using SSH. You have complete root access to it, and you can install relevant programs that benefit your project's purpose, and do work that comply with our Terms of Use.

See Help:Putty for using PuTTY with Windows.

Configuring Project Instances

See Help:Instances#Configuring_Instances

Log your actions

WMCS provides a Server Admin Log which all users can log their project server administration actions on. You can add a log entry in the #wikimedia-cloud connect channel on Freenode by using the sending a message like: !log <projectname> <message>

It's best to log all changes to the instances in your project. Doing so makes it easier for others to follow your progress, and to more easily join your project and help out. See Tool:Stashbot for more information.

Read the documentation and Rules

  • Help:Contents - help table of contents
  • Help:Access - contains information on how to access lab instances
  • Help:Rules - rules you need to follow when using Wikimedia Cloud Services projects
  • Code of Conduct - code of conduct for Wikimedia technical spaces

Help and Participation

If you encounter any problems, drop by our IRC channel at #wikimedia-cloud connect on Freenode. Use the keyword !help to call attention.

Join our mailing lists: announcements, and discussion.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Help:Getting_Started&oldid=1786481"