lcpan tips 022: Testing all dependents

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror.

When you release a new version of a module, CPAN Testers will test your module on a variety of platforms. Sometimes, when they happen to test other modules that depend on your module, they will also uncover problems that is caused by your new version breaking your dependents. However, this is not done systematically or completely. If you want to make sure your changes do not break your dependents, you can test for yourself.

Suppose I’m making some changes to Regexp::Pattern. Let’s see who depends on this module:

% lcpan rdeps Regexp::Pattern -R --phase runtime --rel requires
+----------------------------------+-----------+--------------+-------------+
| dist                             | author    | dist_version | req_version |
+----------------------------------+-----------+--------------+-------------+
| App-Licensecheck                 | JONASS    | v3.0.36      | 0           |
| App-RegexpPatternUtils           | PERLANCAR | 0.003        | 0           |
| Bencher-Scenarios-RegexpPattern  | PERLANCAR | 0.003        | 0           |
| Regexp-Common-RegexpPattern      | PERLANCAR | 0.001        | 0           |
|   Bencher-Scenarios-RegexpCommon | PERLANCAR | 0.02         | 0           |
| Test-Regexp-Pattern              | PERLANCAR | 0.004        | v0.2.7      |
+----------------------------------+-----------+--------------+-------------+

Force reinstalling all dependents

One way to run tests for all dependents is by force-(re)installing all those distributions. One way to do that:

% for mod in `lcpan rdeps Regexp::Pattern -R | td select dist | perl -pe's/^\s+//' | dist2mod`; do
    echo "Installing $mod ..."
    lcpanm --force $mod
  done

dist2mod is provided by App::DistUtils.

Extracting tarballs

Another way to test all dependents is to extract them to a temporary directory then run prove on each. lcpan provides the handy subcommand extract-dist to do this. But you might want to install the dependents all first to pull their dependencies.

% mkdir test-deps
% cd test-deps
% for dist in `lcpan rdeps Regexp::Pattern -R | td select dist`; do
    mkdir $dist && cd $dist && lcpan extract-dist $dist && cd ..
done

We can now test each dependent distro one by one, or all in one go:

% for dist in *; do
    echo "Testing $dist ..."
    cd $dist/*; prove -lr; cd ../..
done
Advertisements

lcpan tips 021: Install CPAN distribution when you only know a script’s name (2)

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror. I find it perfect for my own personal use when working offline.

A previous tip shares this recipe to install a CPAN distribution by mentioning its script name:

% cpanm -n `lcpan script2mod pm-uninstall`

Since then, two convenient utilities have been added: cpanm-script and lcpanm-script. The first one is a thin wrapper over cpanm, while the second one is a thin wrapper over lcpanm (which is itself a thin wrapper over cpanm). They basically work the same as the above recipe: given a script name, look up the distribution name the script is included in then convert it to module name, and then pass it to cpanm.

Being simple thin wrappers, {,l}cpanm-script do not add command-line options of their own but simply convert the script name to module name and pass the rest of the options. For example, when a script with the same name exists in multiple distributions (e.g. hangman) only the first one is chosen to be installed. If you want to install all of them, you can use:

% cpanm -n `lcpan script2mod pm-uninstall`

However, these scripts offer another convenience: tab completion.

% lcpanm-script -
% lcpanm-script hang

It’s better to enter a few characters first to prevent the completion routine from trying to retrieve the whole list of scripts on CPAN.

lcpan tips 020: Finding circular dependency

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror. I find it perfect for my own personal use when working offline.

Before writing Dist::Zilla::Plugin::Prereqs::CheckCircular, from time to time I would end up with circular dependencies in my distributions. For example, I have module A, then write B which depends on A, then write C which depends on B. Later after some hacking I would add C as a dependency to A. Boom, circularity. Installing A becomes impossible because to install A it needs C, which needs B, which needs A… This is usually reported to me by the relentless SREZIC via https://rt.cpan.org, but after a few times I thought it might be a good idea to prevent this from happening in the first place.

The CheckCircular plugin utilizes lcpan to accomplish its goal. First, it gathers the list of modules in the current distribution. Let's say we're building distribution A which contains just a single module A.

Then, the plugin gathers the list of distribution RuntimeRequires prereqs (let's say C D E), then feeds it to lcpan mods --or -x:

% lcpan mods --or -x C D E

This basically filters out modules that do not exist on CPAN because the above commands finds any (--or) module with names exactly matching (-x) the given names. Suppose E is unindexed on CPAN, then the above command will simply output:

C
D

The output of the above command is then fed to lcpan deps -R, which will list all dependencies recursively. Let's say D depends on F and nothing else, while C as we know depends on B, which depends on A. The output of the above command will be:

F
B
  A

The plugin now checks whether in the above list of dependencies, there is any of the current distributions' modules among them. In this case, there is (A). Thus, there is circular dependency and the build is aborted by the plugin.

lcpan tips 019: Helping people find related modules more easily

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror. I find it perfect for my own personal use when working offline.

In a previous tip, about 3 years ago, the output of lcpan related-mods Carp::Always was:

+---------------------------------+-------------------------------------------------------------------+--------------+-----------------------+-----------------------+--------+-----------+
| module                          | abstract                                                          | num_mentions | num_mentions_together | pct_mentions_together | score  | author    |
+---------------------------------+-------------------------------------------------------------------+--------------+-----------------------+-----------------------+--------+-----------+
| Carp::Always::Color             | Carp::Always, but with color                                      | 9            | 4                     | 44.44                 | 711.04 | DOY       |
| Regexp::Debugger                | Visually debug regexes in-place                                   | 7            | 3                     | 42.86                 | 385.74 | DCONWAY   |
| V                               | Print version of the specified module(s).                         | 9            | 3                     | 33.33                 | 299.97 | ABELTJE   |
| App::cpanoutdated               | detect outdated CPAN modules in your environment.                 | 24           | 4                     | 16.67                 | 266.72 | TOKUHIROM |
| Carp::Source::Always            | Warns and dies with stack backtraces and source code context      | 3            | 2                     | 66.67                 | 266.68 | MARCEL    |
| Devel::bt                       | Automatic gdb backtraces on errors                                | 3            | 2                     | 66.67                 | 266.68 | FLORA     |
| Module::Install::AuthorRequires | declare author-only dependencies                                  | 3            | 2                     | 66.67                 | 266.68 | FLORA     |
| MooseX::Types::LoadableClass    | ClassName type constraint with coercion to load the class.        | 12           | 3                     | 25                    | 225    | ETHER     |
| List::AllUtils                  | Combines List::Util and List::MoreUtils in one bite-sized package | 30           | 4                     | 13.33                 | 213.28 | DROLSKY   |
| App::Software::License          | Command-line interface to Software::License                       | 4            | 2                     | 50                    | 200    | ETHER     |
+---------------------------------+-------------------------------------------------------------------+--------------+-----------------------+-----------------------+--------+-----------+

and here's the output of lcpan related-mods Carp::Always --with-scores, today:

+-----------------------------------+----------------------------------------------------------------------------------+--------------+-----------------------+-----------------------+---------+--------------------------------+----------+
| module                            | abstract                                                                         | num_mentions | num_mentions_together | pct_mentions_together | score   | dist                           | author   |
+-----------------------------------+----------------------------------------------------------------------------------+--------------+-----------------------+-----------------------+---------+--------------------------------+----------+
| Carp::Always::Color               | Carp::Always, but with color                                                     | 14           | 10                    | 71.43                 | 7143    | Carp-Always-Color              | DOY      |
| Devel::Confess                    | Include stack traces on all warnings and errors                                  | 12           | 5                     | 41.67                 | 1041.75 | Devel-Confess                  | HAARG    |
| App::cpanoutdated                 | detect outdated CPAN modules in your environment.                                | 24           | 5                     | 20.83                 | 520.75  | cpan-outdated                  | DOLMEN   |
| Devel::bt                         | Automatic gdb backtraces on errors                                               | 2            | 2                     | 100                   | 400     | Devel-bt                       | FLORA    |
| Regexp::Debugger                  | Visually debug regexes in-place                                                  | 9            | 3                     | 33.33                 | 299.97  | Regexp-Debugger                | DCONWAY  |
| V                                 | Print version of the specified module(s).                                        | 9            | 3                     | 33.33                 | 299.97  | V                              | ABELTJE  |
| App::YTDL                         | Download YouTube and other videos.                                               | 3            | 2                     | 66.67                 | 266.68  | App-YTDL                       | KUERBIS  |
| Carp::Source::Always              | Warns and dies with stack backtraces and source code context                     | 3            | 2                     | 66.67                 | 266.68  | Carp-Source                    | MARCEL   |
| Module::Install::AuthorRequires   | declare author-only dependencies                                                 | 3            | 2                     | 66.67                 | 266.68  | Module-Install-AuthorRequires  | FLORA    |
| List::AllUtils                    | Combines List::Util, List::SomeUtils and List::UtilsBy in one bite-sized package | 47           | 5                     | 10.64                 | 266     | List-AllUtils                  | DROLSKY  |
| CPAN::Mini                        | create a minimal mirror of CPAN                                                  | 84           | 6                     | 7.14                  | 257.04  | CPAN-Mini                      | RJBS     |
| MooseX::Types::LoadableClass      | ClassName type constraint with coercion to load the class.                       | 12           | 3                     | 25                    | 225     | MooseX-Types-LoadableClass     | ETHER    |
| App::Nopaste                      | Easy access to any pastebin                                                      | 31           | 4                     | 12.9                  | 206.4   | App-Nopaste                    | ETHER    |
| Clone::Any                        | Select an available recursive-copy function                                      | 4            | 2                     | 50                    | 200     | Clone-Any                      | EVO      |
| Test::Taint                       | Tools to test taintedness                                                        | 4            | 2                     | 50                    | 200     | Test-Taint                     | PETDANCE |
| WWW::YouTube::Download            | Very simple YouTube video download interface                                     | 4            | 2                     | 50                    | 200     | WWW-YouTube-Download           | OALDERS  |
| App::Software::License            | Command-line interface to Software::License                                      | 5            | 2                     | 40                    | 160     | App-Software-License           | ETHER    |
| CPAN::Mini::Devel                 | Create CPAN::Mini mirror with developer releases                                 | 5            | 2                     | 40                    | 160     | CPAN-Mini-Devel                | DAGOLDEN |
| Finance::Currency::Convert::Yahoo | convert currencies using Yahoo                                                   | 5            | 2                     | 40                    | 160     | Finance-Currency-Convert-Yahoo | LGODDARD |
| L                                 | Back end packages (hardware drivers) for Lab::Measurement                        | 5            | 2                     | 40                    | 160     | L                              | SONGMU   |
+-----------------------------------+----------------------------------------------------------------------------------+--------------+-----------------------+-----------------------+---------+--------------------------------+----------+

you can see that, although there are now more false positives, Devel::Confess is now included in the top position. The reason is that Carp::Always and Devel::Confess are mentioned together in more places.

% setop --intersect <(lcpan mentions-for-mod Carp::Always   --json | td select content_path) \
                    <(lcpan mentions-for-mod Devel::Confess --json | td select content_path)
Carp-Always-SyntaxHighlightSource-0.03/lib/Carp/Always/SyntaxHighlightSource.pm
Task-BeLike-RSRCHBOY-0.007/lib/Task/BeLike/RSRCHBOY.pm
Acme-CPANModules-Import-CPANRatings-User-perlancar-0.001/lib/Acme/CPANModules/Import/CPANRatings/User/perlancar.pm
Acme-CPANModules-Import-CPANRatings-User-stevenharyanto-0.001/lib/Acme/CPANModules/Import/CPANRatings/User/stevenharyanto.pm
Acme-CPANModulesBundle-PERLANCAR-0.004/lib/Acme/CPANModules/PERLANCAR/Retired.pm

This helps users that are using or looking at Carp::Always to also find Devel::Confess. (Actually this is not an ideal example, because Carp::Always itself has started to mention Devel::Confess directly, so users can just see the documentation of Carp::Always and find Devel::Confess.)

Many of the above places that mention both modules together are Acme::CPANModules modules, which are just glorified lists. You can help people find related modules by publishing such lists on CPAN, but actually publishing any module or POD which mention related modules together will work equally well.

lcpan tips 018: How did I use lcpan in 2018?

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror. I find it perfect for my own personal use when working offline.

lcpan remains as one of my most-used tools when doing Perl/CPAN development. I use it to search for modules to do some task (usually when I am offline or too lazy to open a browser tab to MetaCPAN, but also to do some grep-ing against the search results). I also use it sometimes to find modules related to a specific module, as keyword searches or seeing the SEE ALSO section of PODs are sometimes insufficient.

The following is a simplistic shell history analysis on how I use lcpan (on my main laptop, at least). The history I have on my laptop is from Nov 2017 to Jan 2019, so that more or less reflects how I used lcpan during 2018.

lcpan and lcpanm

% history | perl -lne's/.+?\]//; s/^pg //; next unless /^lcpanm?\s/; s/(^\S+).*/$1/; print' | freqtable
459	  lcpan
332     lcpanm

I install modules from local mirror using lcpanm quite often.

PAGE_RESULT=1

"pg " is a shell alias which I define as:

alias pg='PAGE_RESULT=1'

because that lets me navigate the output of lcpan with a PAGER (which I have defined as 'less -FRSX').

Most often used subcommands

% history | perl -lne's/.+?\]//; s/^pg //; next unless /^lcpan\s/; s/(^\S+ \S+).*/$1/; print' | freqtable
175	lcpan mods
107	lcpan doc
 35	lcpan scripts
 32	lcpan deps
 17	lcpan rdeps
 16	lcpan related-mods
 14	lcpan src
 11	lcpan stats-last-index-time
 10	lcpan upd
...

I use lcpan most often to:

  • search for modules (lcpan mods);
  • read documentation of modules which I don't have installed yet (lcpan doc);
  • search for scripts (lcpan scripts);
  • see dependencies (lcpan deps);
  • see reverse dependencies (lcpan rdeps);
  • search for related modules (lcpan related-mods);
  • see source code of modules I don't have installed yet (lcpan src);
  • checking the last update date of the index (lcpan stats-last-index-time);
  • updating the mirror/index (lcpan upd which is shortcut for lcpan update).

Finding modules

% history | perl -lne's/.+?\]//; s/^pg //; next unless /^lcpan mods\s/; print' | sort -u
lcpan mods
lcpan mods average
lcpan mods average -l
lcpan mods average|wc -l
lcpan mods binary search -l
lcpan mods bitfin
lcpan mods bitflip
lcpan mods bitgrail
lcpan mods bloomberg
lcpan mods bluetooth
lcpan mods bluetooth -l
lcpan mods b perlstring
lcpan mods ccxt
lcpan mods cpan release
lcpan mods cpan release -l
lcpan mods DateTime::Format
lcpan mods DateTime::Format duration
lcpan mods DateTime::Format::Japanese
lcpan mods datetime iso8601
lcpan mods datetime iso8601 -l
lcpan mods dbi string
lcpan mods dbi string -l
lcpan mods dbi table
lcpan mods dbi table -l
lcpan mods Dist::Zilla::Plugin add -l
lcpan mods Dist::Zilla::Plugin:: --author PERLANCAR -l
lcpan mods Dist::Zilla::Plugin Module
lcpan mods Dist::Zilla::Plugin Module -l
lcpan mods eval
lcpan mods eval -l
lcpan mods fifo -l
lcpan mods File::Slurper
lcpan mods File::Slurper -l
lcpan mods finance crypto
lcpan mods float util
lcpan mods histo
lcpan mods histog -l
lcpan mods histogr -l
lcpan mods histo -l
lcpan mods inside eval -l
lcpan mods interpo
lcpan mods inventory
lcpan mods inventory -l
lcpan mods inventory|wc -l
lcpan mods -l array rank
lcpan mods -l bin groups
lcpan mods -l c encode
lcpan mods -l cli hub
lcpan mods -l compare
lcpan mods -l data cmp
lcpan mods -l dbi csv
lcpan mods -l dbix conn
lcpan mods -l dbix shortcut
lcpan mods -l freq table
lcpan mods -l groups
lcpan mods -l http tiny
lcpan mods -l list rank
lcpan mods -l module abstract
lcpan mods -l module info
lcpan mods -l module pod
lcpan mods -ln role
lcpan mods -l ord
lcpan mods -l ordina
lcpan mods -l ordinaq
lcpan mods -l permute
lcpan mods -l pod abstract
lcpan mods -l Regexp::Common::
lcpan mods -l Regexp::Pattern
lcpan mods -l return level
lcpan mods -l stock exchange
lcpan mods -l test2 tool
lcpan mods -l Test::Approximate
lcpan mods -l test compare
lcpan mods -l Test::Deep::
lcpan mods -l throttle
lcpan mods -l Tickit
lcpan mods -l Tickit Grid
lcpan mods -l tie array
lcpan mods -l Versioning dot
lcpan mods -l who
lcpan mods --namespace Acme::CPANLists
lcpan mods --namespace Acme::CPANLists -l
lcpan mods --namespace Acme::CPANModules -l
lcpan mods --namespace Acme::CPANModuless -l
lcpan mods --namespace Bencher -l
lcpan mods --namespace Bencher::Scenario
lcpan mods --namespace Bencher::Scenario -l
lcpan mods --namespace Data::Sah::Coerce::perl
lcpan mods --namespace Data::Sah::Coerce::perl::str
lcpan mods --namespace DateTime::Format
lcpan mods --namespace DateTime::Format -l
lcpan mods --namespace Graphics::ColorNames
lcpan mods --namespace Graphics::ColorNames|xargs lcpanm -n
lcpan mods --namespace Log::ger
lcpan mods --namespace Log::Ger
lcpan mods --namespace Log::ger|grep -i dump
lcpan mods --namespace String -l
lcpan mods --namespace WordList::Char
lcpan mods -n Archive::Tar
lcpan mods -n digit
lcpan mods -n digit -l
lcpan mods -n digit|wc -l
lcpan mods nearest -l
lcpan mods near -l
lcpan mods -n fifo
lcpan mods -n generic
lcpan mods -n generic -l
lcpan mods -nl Archive::Tar
lcpan mods -nl gen pw
lcpan mods -nl genpw
lcpan mods -nl pass gen
lcpan mods -nl pwd gen
lcpan mods -n pass gen
lcpan mods -n permute digit
lcpan mods Number::Format
lcpan mods Number Format -l
lcpan mods Number::Format -l
lcpan mods pass templat
lcpan mods pass templat -l
lcpan mods percent
lcpan mods percent Sah
lcpan mods percent|wc -l
lcpan mods perinci usage
lcpan mods perl release -l
lcpan mods permute lis
lcpan mods purchase
lcpan mods purchase -l
lcpan mods purchase price
lcpan mods python -l
lcpan mods qr decode
lcpan mods qr decoded
lcpan mods random norm -l
lcpan mods redact
lcpan mods regexp common
lcpan mods regexp common cc
lcpan mods regexp common credit
lcpan mods regexp common -l
lcpan mods ssh client -l
lcpan mods stack trace -l
lcpan mods stock
lcpan mods stock -l
lcpan mods stock|wc -l
lcpan mods Test::Deep:: -l
lcpan mods test path
lcpan mods Text::Histogram
lcpan mods time -l
lcpan mods time of day -l
lcpan mods timeofday -l
lcpan mods version dot
lcpan mods version scheme
lcpan mods version scheme|grep -v Google
lcpan mods who -l
lcpan mods WordList::CryptoCurrency::Catalog::Name

I do keyword searches a lot, and when the keyword is not specific enough I usually add "pg" and "-l" to let me navigate and search further with less. Sometimes I also do namespace searching (--namespace). I search for my own modules a lot, usually because I forget the exact name.

Finding related modules

Modules I tried to find related modules of:

% history | perl -lne's/.+?\]//; next unless /^lcpan related-mods\s/; print' | sort -u

; lcpan related-mods alias::module

lcpan related-mods Data::Diff
lcpan related-mods Data::Throttler
lcpan related-mods Data::Valve
lcpan related-mods IO::Tee
lcpan related-mods Number::Tolerant
lcpan related-mods Package::Alias
lcpan related-mods String::JS
lcpan related-mods Test::Deep
lcpan related-mods utf8

Finding scripts

% history | perl -lne's/.+?\]//; s/^pg //; next unless /^lcpan scripts\s/; print' | sort -u
lcpan scripts bin -l
lcpan scripts count
lcpan scripts count -l
lcpan scripts dateconv
lcpan scripts dateconv -l
lcpan scripts envres
lcpan scripts group -l
lcpan scripts histogram -l
lcpan scripts http-tiny
lcpan scripts interval
lcpan scripts interval -l
lcpan scripts lineno
lcpan scripts linenum
lcpan scripts line number
lcpan scripts lino
lcpan scripts linum
lcpan scripts -l org2html
lcpan scripts -l org-to-html
lcpan scripts parse-nik
lcpan scripts parse-nik -l
lcpan scripts parse num
lcpan scripts _pause
lcpan scripts perl
lcpan scripts perllint
lcpan scripts pick
lcpan scripts pick -l
lcpan scripts pick-l
lcpan scripts rand
lcpan scripts rand -l
lcpan scripts resolution
lcpan scripts resolution -l
lcpan scripts throttle
lcpan scripts zodiac -l

I search for my own scripts a lot too, since I have almost a thousand (~880) of them on CPAN. It's a bit challenging trying to keep the naming organized. When tab completion doesn't help, lcpan comes to the rescue.

lcpan tips 017: Indexing DarkPANs

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror. I find it perfect for my own personal use when working offline.

About DarkPAN

DarkPAN is a term (bonus points for reader who can point out who coined this term) for CPAN-like repository which contains private Perl modules/distributions (or a mix of regular, public CPAN distributions with modified and private ones). Like a CPAN mirror, it is just a hierarchy of files with the main content being Perl distributions put under authors/id/X/XX/ directories, plus a couple of gzip-compressed plaintext indexes in authors/01mailrc.txt.gz (list of author ID's with their names and emails) or authors/00whois.xml and modules/02packages.deails.txt.gz (list of packages found in the distribution files, along with their versions and the files it is found in).

A DarkPAN repository can be created from scratch, or from an existing local mini CPAN mirror. You then "inject" (add) your distribution files to it, not forgetting to update the two abovementioned plaintext index files. I've not kept up-to-date on which tool is now the best to do this, but I still find the good old OrePAN (version 1, last released in 2013 by TOKUHIROM) up to the task. I've tried using the second generation OrePAN2 in the past, but at the time found it to be slow and often reindexing from scratch unnecessarily. I expect things to be better now, but anyway I'll just show you here how to create a DarkPAN using OrePAN:

First you pick a directory name, e.g. /home/perlancar/mypan then just start adding your distribution files to it:

% mkdir /home/perlancar/mypan
% orepan.pl --destination=/home/perlancar/mypan --pause=PERLANCAR Foo-0.001.tar.gz

You need to assign your distribution to an author, which can be whatever (it does not have to be an actual PAUSE ID).

That's it.

Indexing your DarkPAN with lcpan

lcpan can be used to index any CPAN-like repository, so to index your DarkPAN:

% lcpan update --no-use-bootstrap --no-update-files --cpan /home/perlancar/mypan --trace

The --no-use-bootstrap option prevents bootstrapping with the CPAN database index, and --no-update-files prevents downloading files from a CPAN mirror.

To query your DarkPAN, e.g. see list of modules:

% lcpan --cpan /home/perlancar/mypan mods -l

You might want to alias lcpan --cpan /home/perlancar/mypan mods to something shorter, if you type it often enough.

lcpan tips 016: Bootstrap database

About this series: A collection of short blog posts about lcpan tips/recipes. Some posts will also end up in the upcoming App::lcpan::Manual::Cookbook POD to be included in the App::lcpan distribution. First article is here. See the whole series.

About lcpan: an application to download and index a mini CPAN mirror on your local filesystem, so in effect you will have something like your own CPAN with a command-line tool (or perl API) to query and extract information from your mirror. I find it perfect for my own personal use when working offline.

One of the annoying things about using lcpan is how long it takes to create the index the first time (between 5 to 7 hours, depending on your computer's disk and CPU speed). Compared to CPAN::SQLite which only needs about a minute to create its SQLite database, lcpan takes significantly longer time because in addition to parsing authors/01mailrc.txt.gz and modules/02packages.details.txt.gz text files, it also extracts all release tarballs to get the distribution metadata as well as list of files. It also parses each module source code to extract mentions and list of subroutine names. But for that price, you also get more querying capabilities: dependencies, mentions, files inside tarballs, subroutines names, …

To cut down the time and encourage more people to try lcpan out, I've released App::lcpan::Bootstrap, which is just a packaging of a snapshot of the SQLite database. Creating your local index will now just take a few minutes. You need lcpan 1.029 or later to utilize the bootstrap automatically, or you can just decompress the share/db/index.db.xz as your index.db in the CPAN mirror top-level directory.