Main Page: Difference between revisions

From Wikitech
Browse history interactively
← Older edit
Content deleted Content added
VisualWikitext
Ryan Lane (talk | contribs)
2,117 edits
No edit summary
 
(298 intermediate revisions by 72 users not shown)
Line 1: Line 1:
'''Wikitech''' hosts technical documentation for [[foundationsite:|Wikimedia Foundation]] infrastructure and services. To edit this wiki, you need a '''[[Help:Create a Wikimedia developer account|Wikimedia developer account]]'''. Wikitech does not use the [[m:Special:MyLanguage/Help:Unified login|Wikimedia Unified Login]] (like on Wikipedia) and does not permit editing without an account.
== Access rights ==


<templatestyles src="Template:Portal_list_item/styles.css" />
=== Anonymous users ===
<div class="mw-tpl-portal-list">
{{Portal list item
|name=Cloud Services
|image=File:Wikimedia Cloud Services logo with text.svg
|alt=Introduction
|link=Help:Cloud_Services_introduction
|description=Wikimedia Cloud Services (WMCS) provides tools, services, and support to technical contributors who build or maintain software that helps the Wikimedia movement. For help deciding which service is right for you, see the [[Help:Cloud_Services_introduction|Cloud Services introduction]].
{{ContentGrid
|content=
{{Colored box
|title = Toolforge, hosting for tools
|title-link = Portal:Toolforge
|content = Use [[Portal:Toolforge|Toolforge]] to build and host tools (bots, webservices, scheduled jobs) that support wiki projects.
}}
{{Colored box
|title = Data services
|title-link = Portal:Data_Services
|content = Analyze wiki content and data in [[PAWS]], [[m:Research:Quarry|Quarry]], and [[Superset]], or access [[Portal:Data_Services | data services]] from within Cloud VPS or Toolforge.
}}
{{Colored box
|title = Cloud VPS
|title-link = Portal:Cloud_VPS
|content = For advanced users: administer and run your own servers on [[Portal:Cloud_VPS|Cloud VPS]], Wikimedia cloud computing infrastructure powered by [[:w:en:OpenStack|OpenStack]].
}}
}}
}}
{{Portal list item
| name = Production
| image = File:Wikitech-2021-blue-large-icon.svg
| alt = Wikitech logo
| link = Wikimedia infrastructure
| description = [[Wikimedia infrastructure|Wikimedia production]] is the servers, software, and databases that power the live Wikimedia Foundation websites. It is maintained by the Wikimedia Site Reliability Engineering team as well as other teams.
{{ContentGrid
|content=
{{Colored box
|title = MediaWiki at WMF
|title-link = MediaWiki at WMF
|content = Our highly-configured deployment of MediaWiki, the free software wiki platform. To learn how to operate your own MediaWiki site, see [[mw:Special:MyLanguage/Main Page|mediawiki.org]].
}}
{{Colored box
|title = Data centers
|title-link = Data centers
|content = Physical locations where our servers are racked.
}}
{{Colored box
|title = Wikimedia CDN
|title-link = CDN
|content = Traffic routing and HTTP caching for all Wikimedia projects.
}}
}}
}}


</div>
You'll need to have an account created for you. If you currently have SVN access, then you have an account, but need to have it linked to Labs. We are still working out the account activation process, but hope to have it done soon.
<noinclude>__NOTOC__</noinclude>

=== Logged in users ===

After creating an account, you can:

* [[Special:NovaKey|Add and manage your public SSH keys]]

Once you add a key, you'll be able to log into an instance in the main project '''testlabs'''. It may take up to 30 minutes for your key to be propagated to all instances.

If you'd like to create instances, you'll need to have an admin add you to a project, and to the sysadmin role in that project.

You can make [[Resource query examples|queries]] for nova resources; currently Nova instances have semantic properties enabled.

After logging in, you can also access [https://gerrit.wikimedia.org Gerrit]; if you wish to do git checkouts of the puppet repositories, you'll need to log into [https://gerrit.wikimedia.org Gerrit], and add your SSH key there as well. Note: it would be nice if Gerrit could use LDAP for its SSH keystore, instead of its database; I've opened a [http://code.google.com/p/gerrit/issues/detail?id=1124 bug for this], if you'd like to help, please add that feature to Gerrit!

=== Admins ===

==== Wiki Admin ====

If you are a wiki admin, you can:

* [[Special:NovaProject|Create/delete Nova projects and manage project and project role membership]]
* [[Special:NovaRole|Manage global role membership]].
* [[Special:NovaSudoer|Manage sudo policies]].

==== Net Admins ====

If you are a NetAdmin, you can:

* [[Special:NovaAddress|Allocate and associate public IP addresses, and add public DNS entries to instances]]
* [[Special:NovaSecurityGroup|Manage security groups and rules]].

==== Sys Admins ====

If you are a sysadmin, you can:

* [[Special:NovaInstance|Create or manage instances]].
* [[Special:NovaVolume|Create or manage volumes]].

After creating an instance, you'll get an email notifying you that it is ready to be logged into. If you did not add an SSH key prior to creating the instance, you'll need to wait until your key is propagated to the instances (which can take an additional 30 minutes).

==== Cloud Admins ====

In addition to all actions that sysadmins and netadmins, you can:

* [[Special:NovaDomain|Create or manage DNS domains]].

=== Access FAQ ===

* '''Q:''' I was added to a group that gives me access to something in git, but it isn't working, what's wrong?
** '''A:''' Once you have been added to the group, you need to log out of gerrit, then back in. Gerrit pulls its groups from LDAP, but caches them. Logging out, then back in re-synchronizes your groups, and thus clears the cache.

== Git/Gerrit and the puppet repositories ==

Note: Access is currently limited to staff developers and operations engineers. This will change soon.

=== Initial git configuration ===

First, you should set your name and email address in your local git configuration:

git config --global user.email "<email-address>"
git config --global user.name "<full-name>"

=== Checking out the puppet repositories ===

We have the following main repositories and branches:

* operations/puppet
** production (HEAD)
*** used in production
** test
*** used in the testlabs project
* operations/private
** master (HEAD)
*** only used in testlabs project

To check out the production branch of the puppet repo, you can clone it like so:

git clone ssh://<username>@gerrit.wikimedia.org:29418/operations/puppet.git

To check out the test branch of the puppet repo, you can clone it like so:

git clone ssh://<username>@gerrit.wikimedia.org:29418/operations/puppet.git -b test

=== Updating your local repository ===

After you do the initial repository clone, git will automatically set up a remote tracking branch for you. You simply need to fetch/merge or pull:

git fetch
get diff <branch> origin/<branch>
git merge origin/<branch>

or:

git pull

=== Making changes ===

After cloning the repo, you can make changes the normal git way:

<make changes>
git add <newfiles>
git commit -a

After doing so, you'll need to push the changes for review; for instance, here's how to push to the test branch of the puppet repository:

git push ssh://<username>@gerrit.wikimedia.org:29418/operations/puppet HEAD:refs/for/test

You can shorten that by adding a remote:

git remote add puppet ssh://<username>@gerrit.wikimedia.org:29418/operations/puppet

Now you can push for review like so:

git push puppet HEAD:refs/for/test

You can shorten this further by making an alias:

git config alias.push-for-review-test "push puppet HEAD:refs/for/test"

Now you can push for review like so:

git push-for-review-test

Here's the alias for production:

git config alias.push-for-review-production "push puppet HEAD:refs/for/production"

Also, staff operations members have the ability to skip the review step; to do so, simply do a push:

git push puppet

Here's the remotes and aliases for the private repo:

git remote add private ssh://<username>@gerrit.wikimedia.org:29418/operations private
git config alias.push-for-review-private "push puppet HEAD:refs/for/master"

=== Making the changes live in puppet ===

==== Test ====

Changes merged in the test branch will go live in puppet within a minute.

==== Production ====

===== Public repo =====

Changes for production need to be pulled manually; this is done for security purposes. Here's the procedure you should use:

cd ~/puppet
<nowiki>#</nowiki> the next command can be pulled from gerrit; look at the patch set,
<nowiki>#</nowiki> under "Download", use "checkout", and "ssh"
git fetch puppet <refid>
git diff production origin/production | less
git merge origin/production

===== Private repo =====

cd ~/private
<make changes>
git add <newfiles>
git commit -a

===== Private repo =====

cd ~/private
<make changes>
git add <newfiles>
git commit -a

On commit, the changes will go live.

=== Books, guides, tutorials, documentation, etc. ===

* [http://book.git-scm.com/index.html Community git book]
* [http://gerrit.googlecode.com/svn/documentation/2.2.1/index.html Gerrit documentation]
* [http://longair.net/blog/2009/04/16/git-fetch-and-merge/ Why to fetch/merge instead of pull]
* [http://www.ibm.com/developerworks/linux/library/l-git-subversion-1/ Git for Subversion users]

Latest revision as of 19:37, 25 April 2024

Wikitech hosts technical documentation for Wikimedia Foundation infrastructure and services. To edit this wiki, you need a Wikimedia developer account. Wikitech does not use the Wikimedia Unified Login (like on Wikipedia) and does not permit editing without an account.

Introduction
Cloud Services
Wikimedia Cloud Services (WMCS) provides tools, services, and support to technical contributors who build or maintain software that helps the Wikimedia movement. For help deciding which service is right for you, see the Cloud Services introduction.
Toolforge, hosting for tools

Use Toolforge to build and host tools (bots, webservices, scheduled jobs) that support wiki projects.

Data services

Analyze wiki content and data in PAWS, Quarry, and Superset, or access data services from within Cloud VPS or Toolforge.

Cloud VPS

For advanced users: administer and run your own servers on Cloud VPS, Wikimedia cloud computing infrastructure powered by OpenStack.

Wikitech logo
Production
Wikimedia production is the servers, software, and databases that power the live Wikimedia Foundation websites. It is maintained by the Wikimedia Site Reliability Engineering team as well as other teams.
MediaWiki at WMF

Our highly-configured deployment of MediaWiki, the free software wiki platform. To learn how to operate your own MediaWiki site, see mediawiki.org.

Data centers

Physical locations where our servers are racked.

Wikimedia CDN

Traffic routing and HTTP caching for all Wikimedia projects.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Main_Page&oldid=2172155"