Metalink is an Internet standard that harnesses the speed and power of peer to peer networking and traditional downloads with a single click.

For users and content providers, Metalink makes downloading simpler, faster, and more reliable.

Welcome, Google Summer of Code students! If you have any questions at all, please post on our Discussion list or stop in to our IRC channel at irc://irc.freenode.net/metalink (you can contact twanj via private messages there - if no one answers please post on the discussion list).

Metalink was designed for community projects that use multiple mirrors and P2P. There is a whole ecosystem of applications, sites, and projects involved in Metalink.

We invite other Download Managers, Web browsers, BitTorrent clients, and other P2P programs to support Metalink.

If you're a developer, adding Metalink support to Open Source programs and getting projects to use it for their downloads will be a huge help. Examining the source code from current open source Metalink clients and generators could be useful.

Current open source Metalink clients (downloaders)

Current open source Metalink generators:

* Updated to support Metalink 4.0 format.


Ideas:

A number of our previous Ideas have been finished for previous GSoCs for other projects! These ideas could be good starting points for newcomers. They could also be used for GSoC for other projects, for instance Firefox support could be done with Mozilla's Summer of Code. Please familiarize yourself with our project and software, and come up with ideas of your own. Please join our discussion list and post about your ideas! Certain programs that don't yet support Metalink 4.0 could be updated. For smaller, bite sized projects see active tickets.

Open tickets with related projects:
aria2 (C++) bugs and feature requests,
MirrorBrain (Python) issues,
DownThemAll! (JavaScript) issues,
MirrorManager (Python) issues,
Metalink Downloader (JavaScript) issues,
libmetalink bugs (C).


Small (Google Code-In sized) Tasks:

Coding:

Port Metalink Checker from Python 2 to Python 3.
DownThemAll! (Javascript): Implement Directory information in Metalinks.
Improve the Windows port of GGet (Python)
version check (Python or Ruby): Create a short script that downloads a Metalink XML file, reads the version element, and tells you if a newer version has been released.
curl metalink.cgi (Perl): Use Content Disposition header so Metalink file has correct filename.
curl metalink.cgi (Perl): Make sure Metalinks have the correct MIME type,
curl metalink.cgi (Perl): Update so it also creates Meta4 (RFC 5854) files (slightly different format).
MirrorBrain (Python) issues,
Metalink Downloader (JavaScript) issues,
DownThemAll! (JavaScript) issues,
curl (C): use gpgme for automated digital signatures checking.
Update Metalink tools (Unix/Windows, C++) to also support Metalink 4.0 format.
Create a Metalink sharing website
libmetalink (C): file hashing
Or, any small bugs from any Metalink related project!

Documentation/Training:

Documentation/translation in languages other than English of most Metalink programs. Most Metalink programs & their documentation are available in English (and other languages). You would translate the English into your native language (but only if it was not already available).
Work on documentation for aria2's RPC daemon
Improve libmetalink documentation
Create a "Getting Started" guide for people who want to take advantage of Metalink but are average, non-technical people (aka downloaders)
Create a "Getting Started" guide for content producers (people who create files that people would want to download, like ISOs, movies, music, applications, etc)
Create a short intro video explaining Metalink (30-90 seconds?)
Create a longer explanation video (90-180 seconds)

Outreach/Research:

Update our website a bit so it's a bit more friendly and less dense (HTML)
As an outsider, read about Metalink and give us suggestions on what we can do to reach less technical people.
Research package management/updating: a number of updaters like yum and Appupdater already use Metalink. Suggest areas or new apps where we could help.
Contact various projects that use mirrors or have large downloads and let them know about the advantages of Metalink. This would involve posting to public mailing lists or bug trackers.
libmetalink: Need packaging for various distros (Debian)
MirrorBrain & other Metalink apps: Need packaging for various distros (Debian)
metalink-checker & other Python Metalink apps: Get them included on PyPi

Quality Assurance:

Verify aria2 (C++) bugs
Verify DownThemAll! (JavaScript) issues
Test aria2 on multiple platforms (Linux, Windows, OS X, other Unixes)
Test aria2 RPC daemon
Create Metalinks for a test suite. Some Metalinks should be good/correct while others should contain errors (bad mirrors, incorrect checksums, etc) and have the expected outcome listed.

User Interface:

Review aria2 webui, suggest changes. File a bug and create wiki page.
Review DownThemAll! User Interface, suggest changes. File a bug and create wiki page.
Identify parts of the Metalink process that may be confusing or user unfriendly for end users
Identify parts of the Metalink process that may be confusing or user unfriendly for content producers
Review any of the Metalink related apps and suggest UI/UX improvements (Graphics)


Larger Coding Projects (potential Google Summer of Code 2013 sized):

Native Metalink support in Firefox. (full summer)
Importance: High.
Difficulty: Hard.
Skills: C++ coding.
Bonus: Familiarity with Mozilla codebase.
Bug tracker: Feature request.
Details: Firefox already handles XML & checksums so no added dependencies. The goal would be: Single source (ie no multi-source) downloads, checksum verification, and adding multiple files to the download queue. See https://developer.mozilla.org/en/Download_Manager_improvements_in_Firefox_3 (This project was accepted for Mozilla GSoC 2008, but the student also applied to another organization and chose that one).

Native Metalink support in Google Chrome/Chromium. (full summer)
Bug tracker: Feature request.
Importance: High.
Difficulty: Easy to Medium?
Skills: C++ coding.
Details: Chromium code. libxml2 for XML and NSS for checksums, both of which Chrome already depends on. There's also libmetalink. Thread on chromium-discuss. Chrome extension already underway!

Advertise Metalinks w/ Link header, Chunk Checksums, Metalink4 (.meta4 / RFC 5854), in Fedora's MirrorManager (1-2 weeks)
Importance: High.
Difficulty: Easy - Medium
Skills: Python coding.
Details: MirrorManager already supports Metalinks (possibly via a library), but only includes whole file checksums. Chunk checksums (at least for ISOs/large files) would be helpful in repairing downloads.

GMail, Google Code, SourceForge, and Lauchpad support for Metalink
Importance: High.
Difficulty: Easy to Medium?
Skills: More advocacy than coding?
Details: GMail could use metalinks for "Download All Attachments" instead of putting all files in a .ZIP archive. Google Code, SourceForge, and Launchpad all serve up files for many open source projects. Some use mirrors and include checksums and signatures. These are all closed, except for Launchpad, so this involves more advocacy than coding, although maybe Metalink Library for PHP and Python (metalink-library) could be used. SourceForge bug.

Integration with FreeBSD ports infrastructure
Difficulty: Hard.
Contact: Unknown. Contact appropriate maintainers/mailing list to see if this is a possibility.
Details: See this post also. "Metalink is a great concept which might be added to the ports infrastructure. Metalink files can be preprocessed in order to retain big mirror lists in bsd.sites.mk. Or an XML guru can convert bsd.sites.mk to an XML version and include it into metalinks in some way. Stub metalinks can be easily created for each port with a simple script. At this time only www/aria2 has some Metalink support. Aria2 has a few dependencies, but it's completely unnecessary to cram it into the base system. FETCH_DEPENDS would work very nice.Metalink support can be implemented simultaneously with P2P support, which is long overdue."

Metalink integration with Proxy/Cache
Importance: High.
Difficulty: ?
Details: Could be done as a plugin like IntelligentMirror for squid. A proxy could already have files in its cache that are listed in a metalink but they wouldn't be recognized if a different URL is used.


FlashGot support (Firefox Addon).
Importance: Medium, but Native Firefox support is a priority and would make this obsolete.
Difficulty: Medium?
Skills: Javascript coding.
Details: FlashGot is a Firefox Addon that interfaces with external download managers.

Packaging for Linux Distributions!
Importance: High. If apps supporting Metalink exist, but people can't use them...
Details: Most apps could use packaging so people can easily use them. aria2 & KGet are heavily packaged. GGet, Metalink tools, Metalink Editor, Metalink Checker, metalink-library, libmetalink are not packaged as much.

Anything that makes it easier for distributions/projects to automate Metalink creation, like Apache or Drupal modules or plugins.
Difficulty: Easy to Medium.
Skills: Depends on project.
Details: Drupal proposal: posts, blogs, new thread. Peter Poeml has added metalink creation to MirrorBrain, a popular download redirector! Matt Domsch added it to Fedora's MirrorManager.

Custom solutions for specific projects like CPAN, Debian, CentOS, Eclipse, VLC, etc.
Difficulty: Easy to Medium.
Skills: Depends on project.
Details: We'd expect you to do this for multiple projects (5-8), as each will be a small amount of work

Native axel (command line download accelerator) support.
Difficulty: Easy to Medium.
Skills: C coding.
can use libmetalink, a C Metalink Parser, like curl and wget. But this will also require some axel specific code.
Details: Metalink support would require libmetalink or adding XML & checksum library dependency.

Integration with Arch Linux's Pacman updater
Difficulty: Medium to Hard.
Skills: C coding.
Mentor: Tatsuhiro Tsujikawa (aria2 author)
Details: Arch founder Judd Vinet provided an early Metalink script on their forums. A good start for using aria2 can be at Faster Pacman Downloads and Improve Pacman Performance. Mandriva already does something similar.

BitTorrent support for Metalink Checker
Difficulty: Medium to Hard.
Skills: Python coding.
Mentor: Neil McNab (Appupdater / Metalink Checker author)
Details: Metalink Checker currently supports only HTTP and FTP downloads. We should be able to reuse code from other Python based Bittorrent clients and add support to Metalink Checker. BitTornado looks like the best Python option out there. Note that this would then enable Bittorrent support in GGet as well.

A Metalink download system similar to the BitTorrent client / server architecture
Difficulty: Ambitious and extremely hard. :)
Skills: Up to you.
Details: See Ad-Hoc Metalink Swarms?, Mirror URIs & MetaMirror Servers

Integration with Bouncer which serves up files for Mozilla & Gentoo
Difficulty: Medium to Hard.
Skills: PHP
Progress?: Bram submitted a patch which has not been merged yet.
Details: "Bouncer is a database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools." Bouncer already has a database filled with information: hash, urls, preferences, etc. However, the bouncer project is trying to keep it's mirrors in shape by using a spider and checking the downloads against the real files.

By serving up metalinks, you can keep the work of the spider down and give the users more security.

Browser Based Metalink Download Client
Also see: openSUSE GSOC idea
Difficulty: Medium
Skills: Java, some Python coding
Mentor: Neil McNab (Appupdater / Metalink Checker author)
Progress: DLApplet (Java) has been integrated with Metalink Checker using Jython 2.5 RC 3. It should support all features that Metalink Checker does, including multi-threaded download and checksum support. It has been tested with Firefox 3.0.x and IE 8. It is currently very memory intensive (around 80 MB) and a large download for a Java applet (almost 7 MB). This can probably be improved by migrating the Python code to Java since this is likely due to the extra Jython classes. Jython doesn't seem to adapt to applets very well. Maybe adapting Phex (www.phex.org) or JDownloader (http://jdownloader.org/) is a better approach now.
Details: The idea is to create a "clientless" way to download Metalink files. This way anyone with a browser with Java enabled can use and experience Metalinks without having to install a client that supports it. It does not need to be a full featured client, probably only needs to be able to handle one metalink at a time, and has a minimal UI. Can probably leverage reuse of existing Java or Python metalink code (Metalink Checker).
Older Notes: 1. Debian had a GSOC project which is similar, a Javascript Jigdo client. Presumably, it could be adapted to use .metalinks, but it has major limitations. Source code is available here. First, it only works with Firefox right now. Second, because of limitations of javascript you can't grab files from a different domain anyway, limiting its usefulness for metalinks. 2. DLApplet (Java) has been added to the metalinks repository on sourceforge. It is a proof of concept and only supports basic downloads right now (Metalink Lite), it needs multi-threaded download and checksum support. It has been tested with Firefox 3.0.x and IE 7. Maybe try adding JSpeedStreamer as a backend.

High Definition video downloads made faster/error free with Metalinks
Difficulty: Medium to Hard.
Skills: JSpeedStreamer written in Java, Metalink Checker in Python.
Mentor: Neil McNab (Appupdater / Metalink Checker author)
Details: Could integrate with JSpeedStreamer or use Metalink Checker.

"JSpeedStreamer: Multi threaded HTTP proxy to speed up streaming media downloads over HTTP, no more 'Buffering.'JSpeedStreamer is a proxy to speed up streaming media over HTTP by using multiple parallel threads for a single download. It differs from a download manager in that it delivers the data in order so that the resulting stream can be played through MPlayer, VLC or any other media player that can stream over HTTP. JSpeedStreamer receives a HTTP request and then divides up that request into segments, each segment is assigned to a downloading thread and is downloaded using the HTTP Range header. The threads download their data into a circular buffer and the data is delivered to the proxy client - without it knowing that there are multiple threads downloading its request." Metalink Checker: "It wouldn't be too difficult to pipe the output from metalink checker to VLC and see what happens. Of course it will only work if you download the pieces in order (metalink checker generally does) so Bittorrent sources might be out." This also might require using smaller chunk sizes than the default.


Previous Completed Google Summer of Code Projects:


Google Summer of Code(tm) 2012 Projects:

Google Summer of Code(tm) 2012

Chrome extension with Metalink support. (full summer)
Student:Sundaram Ananthanarayanan
Bug tracker: Feature request.
Code repository: Chrome extension
Mentor: Ant Bryan (for Metalink)
Importance: High.
Difficulty: Easy to Medium?
Skills: C++ coding.
Details: Chromium code. libxml2 for XML and NSS for checksums, both of which Chrome already depends on. There's also libmetalink. Thread on chromium-discuss. Chrome extension already underway!

Native wget support.
Student: Ilim Ugur
Bug tracker: Feature request.
Code repository: wget GSoC Project.
Mentor: Micah Cowan (former wget maintainer) via GNU Project
Importance: High.
Difficulty: Hard.
Skills: C coding.
Details: Metalink support single source (or multi-source) downloads, with hopefully the ability to failover to other URLs if a mirror goes down. GNU wget GSoC list.

Metalink/HTTP support in KGet
Student: Aish Raj Dahal
Bug tracker: Feature request.
Code repository: KGet nohistory on github.
Mentor: Lukas Appelhans (from KGet/KDE)
Importance: High.
Difficulty: Easy
Skills: C++ coding.
Details: KGet already supports Metalink/XML, but we're adding Metalink/HTTP support too.

Apache Traffic Server cache hits for download mirrors via Metalink
Student: Jack Bates
Bug tracker: Feature request.
Code repository: In progress!
Mentor: Leif Hedstrom (from Apache Traffic Server)
Importance: High.
Details: Could be done as a plugin like IntelligentMirror for squid. A proxy could already have files in its cache that are listed in a metalink but they wouldn't be recognized if a different URL is used.

Web Client UI for aria2
Student: Hamza Zia (github)
Code repository: webui-aria2
Mentor: Tatsuhiro Tsujikawa (from aria2/Spdylay/Wslay)
Importance: Medium.
Difficulty: Easy
Estimated time: 2-3 weeks
Skills: HTML5, WebSocket, Javascript, CSS.
Details: Utilizing recently added JSON-RPC over WebSocket, now we can build Web Client UI which can directly communicate with aria2 without any intermediate Web frameworks such as PHP. UI: add/remove/pause download and show the progress of downloads and details for individual downloads.

Metalink/HTTP support in DownThemAll!
Student: Hamza Zia (github)
Bug tracker: Feature request / github issue list.
Code repository: downthemall-mirror
Mentor: Nils Maier (from DTA)
Importance: High.
Difficulty: Easy - Medium?
Skills: Javascript coding.
Details: DTA already supports Metalink/XML, adding Metalink/HTTP support should be relatively easy.

GSOC 2009: Included in KDE 4.4. KGet: work on Metalink improvements and integrations with rest of KDE (Nepomuk, Kgpg)
Difficulty: Medium
Skills: C++ coding
Details:Add support for download repair, integration of BitTorrent/FTP/HTTP multisource downloads, pass digital signatures in Metalinks to KGpg, let Nepomuk acquire metadata for semantic desktop. KGet GSoC ideas list.
Progress: Accepted for GSOC 09!

GSOC 2008: GGet uses Metalink Checker for metalink downloads!
 GNOME/GTK GUI for Metalink Checker / Download Manager integrated into the GNOME Desktop
Difficulty: Medium
( Accepted for GNOME GSOC 2008! )
Skills: Python coding, familiarity w/ UI/design issues
Mentor: Neil McNab (Appupdater / Metalink Checker author)
Details: Implement the features of a modern GUI download manager. Metalink Checker is written in Python and can be used as a library.


Previous Completed Projects (Non-GSOC):

Native curl support (included in curl 7.27).
In progress: COMPLETE and added to mainstream curl, but not released yet (scheduled for release in the version after 7.26 in late July 2012).
- (also mulk, based on libcurl, supports metalink). Earlier patch and discussion from 2009.
Implementor: Tatsuhiro Tsujikawa (from aria2/Spdylay/Wslay)
Bounty: $500 ($200 from cURL Project, $300 from Metalink) Importance: HIGHEST!
Difficulty: Hard?
Skills: C coding.
Details: Daniel Stenberg, curl author, writes:"A metalink file transfer library could be made as a layer on top of libcurl, and I think that is the only logical and sensible way.Adding metalink support to the curl tool however, seems like a good idea to me…"
Requirements: (1) Single source (ie not multi-source) downloads from a single URL at a time. (2) Failover to other URLs listed in a Metalink if the current URL is unavailable. (3) Use chunk checksums and full file checksums so downloads retain integrity. (4) Code to curl's coding standards for quality and formatting so it can be incorporated into the curl sources.

Metalink4 (.meta4 / RFC 5854) support in libmetalink (included in libmetalink 0.1.0) (1-2 weeks)
Importance: HIGH! Especially as we finish Metalink support in curl and wget, which both use libmetalink.
Mentor: Tatsuhiro Tsujikawa (from aria2/Spdylay/Wslay/libmetalink)
Implementor: Quentin Stievenart
Difficulty: Easy
Skills: C coding.
Details: Feature details. Implementation details. libmetalink already supports Metalink v3, adding Metalink4 support should be relatively easy.

Metalink/HTTP support in MirrorBrain. (1-2 weeks)
Importance: High.
Difficulty: Easy - Medium
Skills: Python coding.
Details: Feature details. MirrorBrain source code. Foundation for this feature has been added. MB already supports Metalink/XML, Metalink/HTTP should be relatively easy to add.

COMPLETED! Chunk checksum verification in DownThemAll! (Firefox Addon).
Importance: High.
Difficulty: Medium?
Skills: Javascript coding.
Details: DTA does full file verification right now. We just need it to use the chunk checksums to repair any errors. Nils Maier gives details on adding support for chunk checksums to DTA.
Complete: Included in DTA 2.0

Complete: Check out Appupdater 1.0!
Design and create a friendly, easier to use GUI for Appupdater, which provides advanced functionality to Windows, similar to apt-get or yum on Linux.
Difficulty: Easy
Skills: Python coding, familiarity with UI/design issues.
Mentor: Neil McNab (Appupdater / Metalink Checker author)
Details: Get ideas from the better GUI package managers/updaters on Linux.

Complete: Neil has added .jigdo -> .metalink converter/downloader to Metalink Checker.
Jigdo with .metalinks and Metalink downloader.
Difficulty: Easy.
Skills: Bash scripting for Jigdo, C++ for aria2 or other (if needed).
Mentor: Tatsuhiro Tsujikawa (aria2 author)

Complete!: MirrorManager now supports Metalink, added by Matt Domsch.
Integrate Metalink generation into Fedora's MirrorManager
Difficulty: Medium
Skills: Python coding
Details: Coordinate with Matt Domsch, MirrorManager's maintainer. Discussion on fedora-devel & fedora-infrastructure.

Complete: OpenOffice.org serves Metalinks now.
Details: In the beginning of 2010, OpenOffice.org migrated their download infrastructure from Bouncer to MirrorBrain, and solved the issue this way.