What's New in Pyramid 1.3 ========================= This article explains the new features in :app:`Pyramid` version 1.3 as compared to its predecessor, :app:`Pyramid` 1.2. It also documents backwards incompatibilities between the two versions and deprecations added to :app:`Pyramid` 1.3, as well as software dependency changes and notable documentation additions. Major Feature Additions ----------------------- The major feature additions in Pyramid 1.3 follow. Python 3 Compatibility ~~~~~~~~~~~~~~~~~~~~~~ .. image:: python-3.png Pyramid continues to run on Python 2, but Pyramid is now also Python 3 compatible. To use Pyramid under Python 3, Python 3.3 or better is required. Many Pyramid add-ons are already Python 3 compatible. For example, ``pyramid_debugtoolbar``, ``pyramid_jinja2``, ``pyramid_exclog``, ``pyramid_tm``, ``pyramid_mailer``, and ``pyramid_handlers`` are all Python 3-ready. But other add-ons are known to work only under Python 2. Also, some scaffolding dependencies (particularly ZODB) do not yet work under Python 3. Please be patient as we gain full ecosystem support for Python 3. You can see more details about ongoing porting efforts at https://github.com/Pylons/pyramid/wiki/Python-3-Porting . Python 3 compatibility required dropping some package dependencies and support for older Python versions and platforms. See the "Backwards Incompatibilities" section below for more information. The ``paster`` Command Has Been Replaced ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We've replaced the ``paster`` command with Pyramid-specific analogues. Why? The libraries that supported the ``paster`` command named ``Paste`` and ``PasteScript`` do not run under Python 3, and we were unwilling to port and maintain them ourselves. As a result, we've had to make some changes. Previously (in Pyramid 1.0, 1.1 and 1.2), you created a Pyramid application using ``paster create``, like so:: $ $VENV/bin/paster create -t pyramid_starter foo In 1.3, you're now instead required to create an application using ``pcreate`` like so:: $ $VENV/bin/pcreate -s starter foo ``pcreate`` is required to be used for internal Pyramid scaffolding; externally distributed scaffolding may allow for both ``pcreate`` and/or ``paster create``. In previous Pyramid versions, you ran a Pyramid application like so:: $ $VENV/bin/paster serve development.ini Instead, you now must use the ``pserve`` command in 1.3:: $ $VENV/bin/pserve development.ini The ``ini`` configuration file format supported by Pyramid has not changed. As a result, Python 2-only users can install PasteScript manually and use ``paster serve`` instead if they like. However, using ``pserve`` will work under both Python 2 and Python 3. Analogues of ``paster pshell``, ``paster pviews``, ``paster request`` and ``paster ptweens`` also exist under the respective console script names ``pshell``, ``pviews``, ``prequest`` and ``ptweens``. ``paste.httpserver`` replaced by ``waitress`` in Scaffolds ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Because the ``paste.httpserver`` server we used previously in scaffolds is not Python 3 compatible, we've made the default WSGI server used by Pyramid scaffolding the :term:`waitress` server. The waitress server is both Python 2 and Python 3 compatible. Once you create a project from a scaffold, its ``development.ini`` and ``production.ini`` will have the following line:: use = egg:waitress#main Instead of this (which was the default in older versions):: use = egg:Paste#http .. note:: ``paste.httpserver`` "helped" by converting header values that were Unicode into strings, which was a feature that subverted the :term:`WSGI` specification. The ``waitress`` server, on the other hand implements the WSGI spec more fully. This specifically may affect you if you are modifying headers on your responses. The following error might be an indicator of this problem: **AssertionError: Header values must be strings, please check the type of the header being returned.** A common case would be returning Unicode headers instead of string headers. Compatibility Helper Library ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A new :mod:`pyramid.compat` module was added which provides Python 2/3 straddling support for Pyramid add-ons and development environments. Introspection ~~~~~~~~~~~~~ A configuration introspection system was added; see :ref:`using_introspection` and :ref:`introspection` for more information on using the introspection system as a developer. The latest release of the pyramid debug toolbar (0.9.7+) provides an "Introspection" panel that exposes introspection information to a Pyramid application developer. New APIs were added to support introspection :attr:`pyramid.registry.Introspectable`, :attr:`pyramid.config.Configurator.introspector`, :attr:`pyramid.config.Configurator.introspectable`, :attr:`pyramid.registry.Registry.introspector`. ``@view_defaults`` Decorator ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you use a class as a view, you can use the new :class:`pyramid.view.view_defaults` class decorator on the class to provide defaults to the view configuration information used by every ``@view_config`` decorator that decorates a method of that class. For instance, if you've got a class that has methods that represent "REST actions", all which are mapped to the same route, but different request methods, instead of this: .. code-block:: python :linenos: from pyramid.view import view_config from pyramid.response import Response class RESTView(object): def __init__(self, request): self.request = request @view_config(route_name='rest', request_method='GET') def get(self): return Response('get') @view_config(route_name='rest', request_method='POST') def post(self): return Response('post') @view_config(route_name='rest', request_method='DELETE') def delete(self): return Response('delete') You can do this: .. code-block:: python :linenos: from pyramid.view import view_defaults from pyramid.view import view_config from pyramid.response import Response @view_defaults(route_name='rest') class RESTView(object): def __init__(self, request): self.request = request @view_config(request_method='GET') def get(self): return Response('get') @view_config(request_method='POST') def post(self): return Response('post') @view_config(request_method='DELETE') def delete(self): return Response('delete') This also works for imperative view configurations that involve a class. See :ref:`view_defaults` for more information. Extending a Request without Subclassing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is now possible to extend a :class:`pyramid.request.Request` object with property descriptors without having to create a custom request factory. The new method :meth:`pyramid.config.Configurator.set_request_property` provides an entry point for addons to register properties which will be added to each request. New properties may be reified, effectively caching the return value for the lifetime of the instance. Common use-cases for this would be to get a database connection for the request or identify the current user. The new method :meth:`pyramid.request.Request.set_property` has been added, as well, but the configurator method should be preferred as it provides conflict detection and consistency in the lifetime of the properties. Not Found and Forbidden View Helpers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Not Found helpers: - New API: :meth:`pyramid.config.Configurator.add_notfound_view`. This is a wrapper for :meth:`pyramid.config.Configurator.add_view` which provides support for an "append_slash" feature as well as doing the right thing when it comes to permissions (a Not Found View should always be public). It should be preferred over calling ``add_view`` directly with ``context=HTTPNotFound`` as was previously recommended. - New API: :class:`pyramid.view.notfound_view_config`. This is a decorator constructor like :class:`pyramid.view.view_config` that calls :meth:`pyramid.config.Configurator.add_notfound_view` when scanned. It should be preferred over using ``pyramid.view.view_config`` with ``context=HTTPNotFound`` as was previously recommended. Forbidden helpers: - New API: :meth:`pyramid.config.Configurator.add_forbidden_view`. This is a wrapper for :meth:`pyramid.config.Configurator.add_view` which does the right thing about permissions. It should be preferred over calling ``add_view`` directly with ``context=HTTPForbidden`` as was previously recommended. - New API: :class:`pyramid.view.forbidden_view_config`. This is a decorator constructor like :class:`pyramid.view.view_config` that calls :meth:`pyramid.config.Configurator.add_forbidden_view` when scanned. It should be preferred over using ``pyramid.view.view_config`` with ``context=HTTPForbidden`` as was previously recommended. Minor Feature Additions ----------------------- - New APIs: :class:`pyramid.path.AssetResolver` and :class:`pyramid.path.DottedNameResolver`. The former can be used to resolve an :term:`asset specification` to an API that can be used to read the asset's data, the latter can be used to resolve a :term:`dotted Python name` to a module or a package. - A ``mako.directories`` setting is no longer required to use Mako templates Rationale: Mako template renderers can be specified using an absolute asset spec. An entire application can be written with such asset specs, requiring no ordered lookup path. - ``bpython`` interpreter compatibility in ``pshell``. See :ref:`ipython_or_bpython` for more information. - Added :func:`pyramid.paster.get_appsettings` API function. This function returns the settings defined within an ``[app:...]`` section in a PasteDeploy ``ini`` file. - Added :func:`pyramid.paster.setup_logging` API function. This function sets up Python logging according to the logging configuration in a PasteDeploy ``ini`` file. - Configuration conflict reporting is reported in a more understandable way ("Line 11 in file..." vs. a repr of a tuple of similar info). - We allow extra keyword arguments to be passed to the :meth:`pyramid.config.Configurator.action` method. - Responses generated by Pyramid's :class:`pyramid.static.static_view` now use a ``wsgi.file_wrapper`` (see https://www.python.org/dev/peps/pep-0333/#optional-platform-specific-file-handling) when one is provided by the web server. - The :meth:`pyramid.config.Configurator.scan` method can be passed an ``ignore`` argument, which can be a string, a callable, or a list consisting of strings and/or callables. This feature allows submodules, subpackages, and global objects from being scanned. See https://docs.pylonsproject.org/projects/venusian/en/latest/#ignore-scan-argument for more information about how to use the ``ignore`` argument to ``scan``. - Add :meth:`pyramid.config.Configurator.add_traverser` API method. See :ref:`changing_the_traverser` for more information. This is not a new feature, it just provides an API for adding a traverser without needing to use the ZCA API. - Add :meth:`pyramid.config.Configurator.add_resource_url_adapter` API method. See :ref:`changing_resource_url` for more information. This is not a new feature, it just provides an API for adding a resource url adapter without needing to use the ZCA API. - Better error messages when a view callable returns a value that cannot be converted to a response (for example, when a view callable returns a dictionary without a renderer defined, or doesn't return any value at all). The error message now contains information about the view callable itself as well as the result of calling it. - Better error message when a .pyc-only module is ``config.include`` -ed. This is not permitted due to error reporting requirements, and a better error message is shown when it is attempted. Previously it would fail with something like "AttributeError: 'NoneType' object has no attribute 'rfind'". - The system value ``req`` is now supplied to renderers as an alias for ``request``. This means that you can now, for example, in a template, do ``req.route_url(...)`` instead of ``request.route_url(...)``. This is purely a change to reduce the amount of typing required to use request methods and attributes from within templates. The value ``request`` is still available too, this is just an alternative. - A new interface was added: :class:`pyramid.interfaces.IResourceURL`. An adapter implementing its interface can be used to override resource URL generation when :meth:`pyramid.request.Request.resource_url` is called. This interface replaces the now-deprecated ``pyramid.interfaces.IContextURL`` interface. - The dictionary passed to a resource's ``__resource_url__`` method (see :ref:`overriding_resource_url_generation`) now contains an ``app_url`` key, representing the application URL generated during :meth:`pyramid.request.Request.resource_url`. It represents a potentially customized URL prefix, containing potentially custom scheme, host and port information passed by the user to ``request.resource_url``. It should be used instead of ``request.application_url`` where necessary. - The :meth:`pyramid.request.Request.resource_url` API now accepts these arguments: ``app_url``, ``scheme``, ``host``, and ``port``. The app_url argument can be used to replace the URL prefix wholesale during url generation. The ``scheme``, ``host``, and ``port`` arguments can be used to replace the respective default values of ``request.application_url`` partially. - A new API named :meth:`pyramid.request.Request.resource_path` now exists. It works like :meth:`pyramid.request.Request.resource_url` but produces a relative URL rather than an absolute one. - The :meth:`pyramid.request.Request.route_url` API now accepts these arguments: ``_app_url``, ``_scheme``, ``_host``, and ``_port``. The ``_app_url`` argument can be used to replace the URL prefix wholesale during url generation. The ``_scheme``, ``_host``, and ``_port`` arguments can be used to replace the respective default values of ``request.application_url`` partially. - New APIs: :class:`pyramid.response.FileResponse` and :class:`pyramid.response.FileIter`, for usage in views that must serve files "manually". Backwards Incompatibilities --------------------------- - Pyramid no longer runs on Python 2.5. This includes the most recent release of Jython and the Python 2.5 version of Google App Engine. The reason? We could not easily "straddle" Python 2 and 3 versions and support Python 2 versions older than Python 2.6. You will need Python 2.6 or better to run this version of Pyramid. If you need to use Python 2.5, you should use the most recent 1.2.X release of Pyramid. - The names of available scaffolds have changed and the flags supported by ``pcreate`` are different than those that were supported by ``paster create``. For example, ``pyramid_alchemy`` is now just ``alchemy``. - The ``paster`` command is no longer the documented way to create projects, start the server, or run debugging commands. To create projects from scaffolds, ``paster create`` is replaced by the ``pcreate`` console script. To serve up a project, ``paster serve`` is replaced by the ``pserve`` console script. New console scripts named ``pshell``, ``pviews``, ``proutes``, and ``ptweens`` do what their ``paster `` equivalents used to do. All relevant narrative documentation has been updated. Rationale: the Paste and PasteScript packages do not run under Python 3. - The default WSGI server run as the result of ``pserve`` from newly rendered scaffolding is now the ``waitress`` WSGI server instead of the ``paste.httpserver`` server. Rationale: the Paste and PasteScript packages do not run under Python 3. - The ``pshell`` command (see "paster pshell") no longer accepts a ``--disable-ipython`` command-line argument. Instead, it accepts a ``-p`` or ``--python-shell`` argument, which can be any of the values ``python``, ``ipython`` or ``bpython``. - Removed the ``pyramid.renderers.renderer_from_name`` function. It has been deprecated since Pyramid 1.0, and was never an API. - To use ZCML with versions of Pyramid >= 1.3, you will need ``pyramid_zcml`` version >= 0.8 and ``zope.configuration`` version >= 3.8.0. The ``pyramid_zcml`` package version 0.8 is backwards compatible all the way to Pyramid 1.0, so you won't be warned if you have older versions installed and upgrade Pyramid itself "in-place"; it may simply break instead (particularly if you use ZCML's ``includeOverrides`` directive). - String values passed to :meth:`pyramid.request.Request.route_url` or :meth:`pyramid.request.Request.route_path` that are meant to replace "remainder" matches will now be URL-quoted except for embedded slashes. For example:: config.add_route('remain', '/foo*remainder') request.route_path('remain', remainder='abc / def') # -> '/foo/abc%20/%20def' Previously string values passed as remainder replacements were tacked on untouched, without any URL-quoting. But this doesn't really work logically if the value passed is Unicode (raw unicode cannot be placed in a URL or in a path) and it is inconsistent with the rest of the URL generation machinery if the value is a string (it won't be quoted unless by the caller). Some folks will have been relying on the older behavior to tack on query string elements and anchor portions of the URL; sorry, you'll need to change your code to use the ``_query`` and/or ``_anchor`` arguments to ``route_path`` or ``route_url`` to do this now. - If you pass a bytestring that contains non-ASCII characters to :meth:`pyramid.config.Configurator.add_route` as a pattern, it will now fail at startup time. Use Unicode instead. - The ``path_info`` route and view predicates now match against ``request.upath_info`` (Unicode) rather than ``request.path_info`` (indeterminate value based on Python 3 vs. Python 2). This has to be done to normalize matching on Python 2 and Python 3. - The ``match_param`` view predicate no longer accepts a dict. This will have no negative affect because the implementation was broken for dict-based arguments. - The ``pyramid.interfaces.IContextURL`` interface has been deprecated. People have been instructed to use this to register a resource url adapter in the "Hooks" chapter to use to influence :meth:`pyramid.request.Request.resource_url` URL generation for resources found via custom traversers since Pyramid 1.0. The interface still exists and registering an adapter using it as documented in older versions still works, but this interface will be removed from the software after a few major Pyramid releases. You should replace it with an equivalent :class:`pyramid.interfaces.IResourceURL` adapter, registered using the new :meth:`pyramid.config.Configurator.add_resource_url_adapter` API. A deprecation warning is now emitted when a ``pyramid.interfaces.IContextURL`` adapter is found when :meth:`pyramid.request.Request.resource_url` is called. - Remove ``pyramid.config.Configurator.with_context`` class method. It was never an API, it is only used by ``pyramid_zcml`` and its functionality has been moved to that package's latest release. This means that you'll need to use the 0.9.2 or later release of ``pyramid_zcml`` with this release of Pyramid. - The older deprecated ``set_notfound_view`` Configurator method is now an alias for the new ``add_notfound_view`` Configurator method. Likewise, the older deprecated ``set_forbidden_view`` is now an alias for the new ``add_forbidden_view`` Configurator method. This has the following impact: the ``context`` sent to views with a ``(context, request)`` call signature registered via the ``set_notfound_view`` or ``set_forbidden_view`` will now be an exception object instead of the actual resource context found. Use ``request.context`` to get the actual resource context. It's also recommended to disuse ``set_notfound_view`` in favor of ``add_notfound_view``, and disuse ``set_forbidden_view`` in favor of ``add_forbidden_view`` despite the aliasing. Deprecations ------------ - The API documentation for ``pyramid.view.append_slash_notfound_view`` and ``pyramid.view.AppendSlashNotFoundViewFactory`` was removed. These names still exist and are still importable, but they are no longer APIs. Use ``pyramid.config.Configurator.add_notfound_view(append_slash=True)`` or ``pyramid.view.notfound_view_config(append_slash=True)`` to get the same behavior. - The ``set_forbidden_view`` and ``set_notfound_view`` methods of the Configurator were removed from the documentation. They have been deprecated since Pyramid 1.1. - All references to the ``tmpl_context`` request variable were removed from the docs. Its existence in Pyramid is confusing for people who were never Pylons users. It was added as a porting convenience for Pylons users in Pyramid 1.0, but it never caught on because the Pyramid rendering system is a lot different than Pylons' was, and alternate ways exist to do what it was designed to offer in Pylons. It will continue to exist "forever" but it will not be recommended or mentioned in the docs. - Remove references to do-nothing ``pyramid.debug_templates`` setting in all Pyramid-provided .ini files. This setting previously told Chameleon to render better exceptions; now Chameleon always renders nice exceptions regardless of the value of this setting. Known Issues ------------ - As of this writing (the release of Pyramid 1.3b2), if you attempt to install a Pyramid project that used the ``alchemy`` scaffold via ``setup.py develop`` on Python 3.2, it will quit with an installation error while trying to install ``Pygments``. If this happens, please just rerun the ``setup.py develop`` command again, and it will complete successfully. This is due to a minor bug in SQLAlchemy 0.7.5 under Python 3, and has been fixed in a later SQLAlchemy release. Keep an eye on https://github.com/sqlalchemy/sqlalchemy/issues/2421 Documentation Enhancements -------------------------- - The :ref:`bfg_sql_wiki_tutorial` has been updated. It now uses ``@view_config`` decorators and an explicit database population script. - Minor updates to the :ref:`bfg_wiki_tutorial`. - A narrative documentation chapter named :ref:`extconfig_narr` was added; it describes how to add a custom :term:`configuration directive`, and how use the :meth:`pyramid.config.Configurator.action` method within custom directives. It also describes how to add :term:`introspectable` objects. - A narrative documentation chapter named :ref:`using_introspection` was added. It describes how to query the introspection system. - Added an API docs chapter for :mod:`pyramid.scaffolds`. - Added a narrative docs chapter named "Creating Pyramid Scaffolds". - Added a description of the ``prequest`` command-line script at :ref:`invoking_a_request`. - Added a section to the "Command-Line Pyramid" chapter named :ref:`making_a_console_script`. - Removed the "Running Pyramid on Google App Engine" tutorial from the main docs. It survives on in the Pyramid Community Cookbook as :ref:`Pyramid on Google's App Engine (using appengine-monkey) `. Rationale: it provides the correct info for the Python 2.5 version of GAE only, and this version of Pyramid does not support Python 2.5. - Updated the :ref:`changing_the_forbidden_view` section, replacing explanations of registering a view using ``add_view`` or ``view_config`` with ones using ``add_forbidden_view`` or ``forbidden_view_config``. - Updated the :ref:`changing_the_notfound_view` section, replacing explanations of registering a view using ``add_view`` or ``view_config`` with ones using ``add_notfound_view`` or ``notfound_view_config``. - Updated the :ref:`redirecting_to_slash_appended_routes` section, replacing explanations of registering a view using ``add_view`` or ``view_config`` with ones using ``add_notfound_view`` or ``notfound_view_config`` - Updated all tutorials to use ``pyramid.view.forbidden_view_config`` rather than ``pyramid.view.view_config`` with an HTTPForbidden context. Dependency Changes ------------------ - Pyramid no longer depends on the ``zope.component`` package, except as a testing dependency. - Pyramid now depends on the following package versions: zope.interface>=3.8.0, WebOb>=1.2dev, repoze.lru>=0.4, zope.deprecation>=3.5.0, translationstring>=0.4 for Python 3 compatibility purposes. It also, as a testing dependency, depends on WebTest>=1.3.1 for the same reason. - Pyramid no longer depends on the ``Paste`` or ``PasteScript`` packages. These packages are not Python 3 compatible. - Depend on ``venusian`` >= 1.0a3 to provide scan ``ignore`` support. Scaffolding Changes ------------------- - Rendered scaffolds have now been changed to be more relocatable (fewer mentions of the package name within files in the package). - The ``routesalchemy`` scaffold has been renamed ``alchemy``, replacing the older (traversal-based) ``alchemy`` scaffold (which has been retired). - The ``alchemy`` and ``starter`` scaffolds are Python 3 compatible. - The ``starter`` scaffold now uses URL dispatch by default.