Help:Getting Started: Difference between revisions

From Wikitech
Content deleted Content added
Remove content duplicated elsewhere and make this page mostly navigational
Β 
(47 intermediate revisions by 11 users not shown)
Line 1: Line 1:
{{Mbox|image=[[File:Ambox PR.svg|40x40px|link=|alt=]]|text=Please read the [[Help:Cloud_Services_Introduction|Wikimedia Cloud Services Introduction]].}}


== Set up your accounts ==
{{TOC|align=right}}


{{Account_setup}}
==Overview==


== Get started with Toolforge ==
Cloud Services provides tools, services, and support for technical collaborators who want to contribute to Wikimedia software projects. This page outlines basic first steps to making contributions to this wiki and Wikimedia's technical spaces.


Follow the steps on [[Help:Toolforge/Quickstart]].
'''This guide will teach you to:'''


== Get started with Cloud VPS Projects ==
* Create a Wikimedia developer account for Toolforge, Cloud VPS, and general use
* Begin using Gerrit for code review
* Access an existing project or begin a new project


Follow the steps on [[Portal:Cloud VPS]].
=={{anchor|Create_a_User_Account}} Step 1: Create a Wikimedia developer account==


== Get started with Gerrit ==
Most of Wikimedia's technical spaces use a Wikimedia developer account. A developer account is required to:


Gerrit is Wikimedia's code review system.
* Edit this wiki
* Work with [[toollabs:openstack-browser/project/|Cloud VPS Projects]]
* Request access to [[Portal:Toolforge|Toolforge]]
* Login to [https://horizon.wikimedia.org/auth/login/?next=/ the Cloud VPS management console (Horizon)]
* Login to [[phab:| the bug tracking system used for Wikimedia projects (Phabricator)]]
* Commit code to Wikimedia’s [[w:Git_(software)|Git]] repositories via [[w:Gerrit_(software)|our code review system (Gerrit)]]
* Access other protected Wikimedia services, after signing a [[Volunteer NDA]] and being approved for access.


Once you have [[Help:Create_a_Wikimedia_developer_account|created your Wikimedia account and Wikimedia developer account]], including your UNIX shell username, follow the instructions in the [https://www.mediawiki.org/wiki/Gerrit/Tutorial#Set_Up_SSH_Keys_in_Gerrit Gerrit tutorial] to get started.
==={{anchor|Tools Users}} Toolforge users ===
==== Creating an account ====
# Use the '''Toolforge admin console''' to [https://toolsadmin.wikimedia.org/register/ create a new developer account].
#* Make careful note of the ''LDAP username'' and ''UNIX shell username'' you choose
#** Your LDAP username and password will be used to login here on Wikitech and on [[:mw:Gerrit|Gerrit]] and the [https://toolsadmin.wikimedia.org/ Toolforge admin console].
#** Your shell name will be used when you interact with servers via [[Help:Access|ssh]] and when using [[:mw:Gerrit/Getting_started|git with Gerrit]].
# [https://toolsadmin.wikimedia.org/tools/membership/apply Fill out an access request to join the Toolforge project].
# [[:mw:Gerrit/Tutorial#Generate_a_new_SSH_key|Generate an SSH key]] that you will use along with your ''shell'' account name to log into the Toolforge servers if you do not already have one.
# Associate your SSH public key with your Wikitech account using the [[Special:Preferences#mw-prefsection-openstack|Preferences > OpenStack tab]] or [https://toolsadmin.wikimedia.org/profile/settings/ssh-keys Toolforge admin console].
# Wait for your access request for the Tools project to be processed (you should receive messages on your [[Special:MyTalk | Wikitech talk page]]).


{{:Help:Cloud Services communication}}
==== After account creation ====
* Use SSH to login to Toolforge. In a command-line terminal, enter: <kbd>ssh -i ~/.ssh/id_rsa ''username''@login.tools.wmflabs.org</kbd> (''username'' is the "Instance ''shell'' account name" you specified when you created an account. If you are running Windows you will need to use an ssh client like [[Help:Access to Toolforge instances with PuTTY and WinSCP|puTTY]].
* Use SSH-based utilities such as scp and sftp to transfer files between Toolforge and your computer.
* Access MySQL from SSH (e.g. <kbd>sql enwiki</kbd> is a shortcut command to connect to the copy of English Wikipedia)
* Create tools (see {{section link|Help:Toolforge|Creating a new Tool account}}).


[[Category:Cloud Services]]
==== Common issues ====
* Your LDAP username and your shell login username may be different. Visit [https://toolsadmin.wikimedia.org/profile/settings/accounts Settings > Linked accounts in the Toolforge admin console] and check your "Shell account name".
* The passwords you chose for your LDAP account and SSH key may be different.
* When you login with SSH you are in your personal folder. To quickly go to your tool account enter: <kbd>become ''tool_name''</kbd>
* If you login with WinSCP you will transfer files as yourself, not as your tool. The UNIX group for both accounts is the same so you can make files editable by all maintainers of the tool with:
:<syntaxhighlight lang="shell-session">
$ become TOOL_NAME
$ chmod --recursive g+rw .
</syntaxhighlight>
* The web service for your tool is not started by default. To start it enter: <kbd>webservice start</kbd>
* When doing file system intensive tasks (like <code>git clone</code>), it may be useful to perform the task on a host-local filesystem (like <code>/tmp</code>) to avoid the speed penalty of NFS. For example, cloning git repositories in <code>/tmp</code> and then moving it to the location you actually need it in can be much faster than cloning it in the original location.
** Note: the speed penalty can be severe. For example, running ''wc -l'' on a 285 MB text file took 0:01 on /tmp and 3:14 over NFS.
* If you need to use the server for a long time, and want to execute a task even when you're not connected to the ssh, you can use <code>screen</code>. Note that screen needs to be run before <code>become TOOL_NAME</code>. Screen '''should not''' be used to run long running scripts. These should be [[Help:Toolforge/Grid|run as grid jobs]] instead.
* You might need to use <code>ssh-add</code> on your local computer after creating a new key.

==== Using multiple ssh agents ====
If you use multiple <tt>ssh-agent</tt>s (to connect to your personal or company system, for example), see [[SSH access]] for more information about setting up a primary and a Cloud VPS agent.

=== VPS and General Users ===
==== Creating an account ====
Create a Wikimedia developer account on Wikitech [https://wikitech.wikimedia.org/w/index.php?title=Special:UserLogin&returnto=Help%3AGetting+Started&type=signup using the account creation form].

==== Uploading an SSH key {{anchor|Upload_your_SSH_key}} ====
Wikimedia Cloud Services products and [[gerrit:|Wikimedia Code Review]] use an [[w:Secure_Shell|SSH key]] to authenticate you. If you already have an SSH key you can use that one, or [[mw:Git/Tutorial#Set_Up_SSH_Keys_in_Gerrit|generate a new one]] specifically for Wikimedia use; the default public key name generated by [[w:Ssh-keygen|ssh-keygen]] is typically <tt>id_rsa.pub</tt>.

Upload the generated public key from the [[Special:Preferences#mw-prefsection-openstack|'{{MediaWiki:Prefs-openstack}}' tab in your Preferences]] '''or''' use [https://toolsadmin.wikimedia.org/profile/settings/ssh-keys the toolsadmin interface].

See Also: [[Help:SSH|SSH Help]]

You can test your key by attempting to connect to the <code>bastion</code> VPS project: <code>ssh -v bastion.wmflabs.org</code>.

{{note|You will only be able to ssh to <code>bastion</code> '''after''' you have been added as a member of at least one VPS project (e.g. Toolforge). Users can verify their project membership via OpenStack Browser at <code>https<nowiki>:</nowiki>//tools.wmflabs.org/openstack-browser/user/<shellname></code>}}

==== Uploading your key to Wikimedia's Gerrit (Code Review) ====
Upload your public key to [https://gerrit.wikimedia.org Gerrit (Wikimedia's code review tool)].
# β€œSign In” with your Wikimedia Developer Account
# Click β€œSettings” link in drop down menu under your username at page top right
# Click β€œSSH Public Keys” in left sidebar
# Paste your public key into the text field and click β€œAdd” button. Thereafter, the form presents "Add key ..." button instead, for if you have more keys you want to add.

=== Gerrit/Git committers ===
If you are just simply getting an account for committing into [[w:Gerrit_(software)|Gerrit]], read some detailed documentation about it
* [[mw:Gerrit]] - mediawiki.org's guide to using Gerrit
* [[Help:Git rebase]] - Because the importance of using fetch and rebase (instead of pull or merge) can't be over-emphasized.

== Getting started with Toolforge ==
=== Requesting access ===
Once you have created a developer account, you must request access to the [[Portal:Toolforge|Toolforge]] project by submitting a [https://toolsadmin.wikimedia.org/tools/membership/apply Toolforge project membership request].

Requests for access are generally dealt with within days, though response-time may be longer depending on admin availability. If you need immediate assistance, please contact us on the {{freenode|wikimedia-cloud}} IRC channel.

=== Receiving access to the 'tools' project ===
Once your 'tools' project access request has been processed, you will become a member of the [[Portal:Toolforge|Toolforge]] Cloud VPS project, and will be able to access it using the "UNIX shell username" provided when creating your developer account and the SSH private key matching the public key you supplied for authentication. For more information about accessing the project, please see [[Help:Toolforge#Using_Toolforge_and_managing_your_files|Using Toolforge and managing your files]].

You will be notified on Wikitech that your user rights were changed and that you have been added to the project [[Nova Resource:Tools]]. In other words, your Toolforge account is ready for you to use!

== Getting started with VPS Projects ==
=== Join a Project ===
In order to work on a Cloud VPS project you need to be granted project membership by a project administrator.

* Find the project you wish to join using [[toollabs:openstack-browser/|OpenStack browser]].
** To join Toolforge, see the instructions in [[Help:Getting Started#Getting_started_with_ToolForge|Getting started with Toolforge]]
** If you want to join another project, ask a project administrator for that project. You can find the list of project admins by going to <kbd><nowiki>http://tools.wmflabs.org/openstack-browser/project/<project-name></nowiki></kbd>

=== Add members and admin users to a Project ===
* Project admins can go to [[Special:NovaProject]] and add people by clicking the 'Add member' link.
* Project admins can also grant other members admin rights from the [[Special:NovaProject]] page by clicking the 'Add member' link for the 'projectadmin' role in the Role column.
* Log at {{irc|wikimedia-cloud}} that you added/granted admin to the user. See [[Help:Getting Started#Log_your_actions]] for how to add to the admin log from irc.

=== Request creation of a new Project ===
If you need to create a new project go to the [[phab:project/view/2875/|"Cloud-VPS (Project-requests)" phabricator task]] and follow the instructions it provides.

=== Project Instances ===
==== Creating Instances ====
To create instances within a project, see [[Help:Instances]]

==== Access Project Instances ====
After you have been [[#Projects|added to or created a project]] and successfully [[#Upload_your_SSH_key|added your SSH key]] to Wikitech and Wikimedia Code Review, read the [[Help:Access]] manual.

===== Using ProxyCommand for SSH =====
The easiest way to access your project instance is using the <code>ProxyCommand</code> ssh option to directly connect to your instance.

* Follow the steps at [[Help:Access#Accessing_instances_with_ProxyCommand_ssh_option_.28recommended.29|set up the ProxyCommand ssh option]] first
* Connect to your instance using the following command from your local computer
:<syntaxhighlight lang="shell-session">
$ ssh <instance-name>.<project-name>.eqiad.wmflabs
</syntaxhighlight>

The instance name is given in the "Instance name" column in [[Special:NovaInstance]]. Logging in to the instance is just like logging into any computer using SSH. You have complete root access to it, and you can install relevant programs that benefit your project's purpose, and do work that comply with our Terms of Use.

See [[Help:Putty]] for using PuTTY with Windows.

==== Configuring Project Instances ====
See [[Help:Instances#Configuring_Instances]]

=== Log your actions ===
WMCS provides a [[Server Admin Log]] which all users can log their project server administration actions on. You can add a log entry in the {{irc|wikimedia-cloud}} channel on [[:en:Freenode|Freenode]] by using the sending a message like: <code>!log <projectname> <message></code>

It's best to log all changes to the instances in your project. Doing so makes it easier for others to follow your progress, and to more easily join your project and help out. See [[Tool:Stashbot]] for more information.

== Read the documentation and Rules ==
* [[Help:Contents]] - help table of contents
* [[Help:Access]] - contains information on how to access lab instances
* [[Help:Rules]] - rules you need to follow when using Wikimedia Cloud Services projects
* [[mw:Code of Conduct|Code of Conduct]] - code of conduct for Wikimedia technical spaces

== Help and Participation ==
If you encounter any problems, drop by our [[Help:IRC|IRC channel]] at {{irc|wikimedia-cloud}} on [[:en:Freenode|Freenode]]. Use the keyword <kbd>!help</kbd> to call attention.

Join our mailing lists: [https://lists.wikimedia.org/mailman/listinfo/cloud-announce announcements], and [https://lists.wikimedia.org/mailman/listinfo/cloud discussion].

[[Category:Documentation]]

Latest revision as of 20:14, 22 September 2023

Set up your accounts

To access Toolforge, Cloud VPS, or other Wikimedia developer tools, you need the following two types of accounts:

Account Type Description Where to sign up
Wikimedia account Also known as "Wikimedia single user login (SUL)" account. Allows you to log into general wikis like Wikipedia, MediaWiki, and MetaWiki. Create Wikimedia account
Wikimedia developer account Username (also called LDAP username) is used to log in on Wikitech (this wiki), Gerrit, and the Toolforge admin console. In GitLab it is your displayed name, so you may want it to be your full name. You will not be able to change this later!

Shell username is used to log into Toolforge, other Wikimedia VPS or Wikimedia production hosts using SSH.

Create Wikimedia developer account

Get started with Toolforge

Follow the steps on Help:Toolforge/Quickstart.

Get started with Cloud VPS Projects

Follow the steps on Portal:Cloud VPS.

Get started with Gerrit

Gerrit is Wikimedia's code review system.

Once you have created your Wikimedia account and Wikimedia developer account, including your UNIX shell username, follow the instructions in the Gerrit tutorial to get started.

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support
Stay aware of critical changes and plans
Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)