Introduction

Parse2Plone is a "one off" HTML parser (in the form of a Buildout recipe that creates a script for you) to easily get content from static HTML websites (on the file system) into Plone.

It is designed to satisfy only the most trivial of use cases (e.g. a simple static site in /var/www/html); for more serious deployments, you might try funnelweb.

Warning

This is a Buildout recipe for use with Plone; by itself it does nothing. If you don't know what Plone is, please see: http://plone.org. If you don't know what Buildout is, please see: http://www.buildout.org/.

Getting started

Because it drives the author nuts whenever he has to dig for a recipe's options, here are this recipe's options:

[import]
recipe = parse2plone

# core features
path = /Plone
illegal_chars = _ .
html_extensions = html
image_extensions = gif jpg jpeg png
file_extensions = mp3 xls
target_tags = a div h1 h2 p

# additional bells & whistles
force = false
publish = false
collapse = false
create_spreadsheet = false
replacetypes =
rename =
match =
paths =

Justification

Why did you create Parse2Plone when collective.transmogrifier already existed?

Here are some reasons:

• Because Parse2Plone is aimed at lowering the bar for folks who don't already know (or want to know) what a "transmogrifier blueprint" is but are able to update their buildout.cfg file; run Buildout; then run a single command; all without having to think too much.
• collective.transmogrify provides a framework for creating reusable pipes (whose definitions are called blueprints). Parse2Plone provides a single, non-reusable script.
• Transmogrifier and friends appear to be "developer's tools", while the author wants Parse2Plone to be an "end user's tool".

If you are a developer looking to create repeatable migrations, you probably want to be using collective.transmogrifier. If you are an end user that just wants to see your static website in Plone, then you might want to give Parse2Plone a try.

There is also this user/contributor comment, which captures the author's sentiment:

Parse2Plone's release was very timely as I need either this or something very
similar - and while I've no doubt I could make transmogrify do the job, it's a
lot of work for a one-shot loading of legacy pages.

                                                -Derek Broughton, Pointer
                                                 Stop Consulting, Inc.

That's great, but why didn't you try to improve collective.transmogrifier and make it more user friendly?

Here are some reasons:

• The author is a minimalist. One of the design aims of Parse2Plone was to have as few dependencies as possible. There are two "big" dependencies, zc.buildout and lxml. Using Buildout was a design compromise, and using lxml is pretty much a must if you want to "parse". Oh, and BeautifulSoup was thrown in after the author read this: http://codespeak.net/lxml/elementsoup.html
• parse2plone in addition to its primary role as a content importer, is intended to serve as an educational tool; both for the author and consumer. Python coding best practices, and demonstrating how to script tasks in Plone with "bin/instance run" are the aim. Forking and pull requests are encouraged.
• The author had an itch to scratch; it will be nice for him to be able to say "just go write a script" and then point to an example.

All of that said, the author understands that reusability and conservation of developer resources are important goals, especially for the Plone project. But for better or worse, these were not the goals of Parse2Plone. However, you can be sure that the lessons learned while developing Parse2Plone will be applied outside of it, particularly with regard to the conservation of developer resources within the Plone project.

Installation

You can install Parse2Plone by editing your buildout.cfg file like so. First add an import section:

[import]
recipe = parse2plone

Then add the import section to the list of parts:

[buildout]
...
parts =
    ...
    import

Now run bin/buildout as usual.

Execution

Now you can run Parse2Plone like this:

$ bin/plone run bin/import /path/to/files

Example

If you have a site in /var/www/html that contains the following:

/var/www/html/index.html
/var/www/html/about/index.html

You should run:

$ bin/plone run bin/import /var/www/html

And the following will be created:

• http://localhost:8080/Plone/index.html
• http://localhost:8080/Plone/about/index.html

Customization

Modifying the default behavior of parse2plone is easy; just use the command line options or add parameters to your buildout.cfg file. Both approaches allow customization of the exact same set of options, but the command line arguments will trump any settings found in your buildout.cfg file.

[import]
recipe = parse2plone
path = /Plone/foo
html_extensions = htm
image_extensions = png
target_tags = p

$ bin/plone run bin/import /path/to/files --path=/Plone/foo

$ bin/plone run bin/import /path/to/files --html-extensions=htm

$ bin/plone run bin/import /path/to/files --image-extensions=png

$ bin/plone run bin/import /path/to/files --file-extensions=pdf

$ bin/plone run bin/import /path/to/files --target-tags=p

$ bin/plone run bin/import --paths=sample:Plone/sample,sample2:Plone/sample2

$ bin/plone run bin/import --create-spreadsheet /var/www/html

$ bin/plone run bin/import -h

$ bin/plone run bin/import /path/to/files -p /Plone/foo --html-extensions=html \
    --image-extensions=png --target-tags=p

Troubleshooting

Here are some trouble-shooting comments/tips.

Contact

Questions, comments, or concerns? Please e-mail: aclark@aclark.net.

Credits

Development sponsored by Radio Free Asia

History