Status and Roadmap

Looking at a test version of 1.9.rc3, the Newsletter feature seems to have all the key requirements for a simple broadcast mailing list:

• Web form subscription
• Optional verification
• Email link opt-out
• Subscribe entire user groups
• Admin can add and remove subscribers
• Web based message submission form

But there are many issues to be addressed:

• Terminology refinement
• Groups integration
• Bounce handling
• General usability
• And many wish list items

There is also talk of doing a major upgrade to support two-way discussions. Others are keen on replacing this feature with an external mailing list application like PHPlist, Sympa or Mailman. Integration with wiki is also an interesting concept.

The plan now is to decide on what modications are required to make the feature usable in 1.9 and what would make it optimal in version 1.10.

And Damian reminds us: You cannot remove a feature, that is not preserving the environment, see 3Rules.

Requested Upgrades for 1.9 Release

Bugs to Fix

• Failed opening lang//language.php when subscription was supposed to be confirmed.
  Warning: Failed opening 'lang//language.php' for inclusion (include_path='lib/pear:lib/adodb:lib/pear:lib:.:lib:lib/pear:/usr/share pear/') in /home/avantech/multiki19/lib/smarty_tiki/prefilter.tr.php on line 22

Refine use of terminology

I suggest replacing the term "Newsletters" with "Mailing List" since some people may use the feature in ways that are not consistent with the idea of a newsletter, plus user organizations may already have real print newsletters. For clarity sake, I use “newsletter” throughout these recommendations, but they could be replaced with “mailing list”.

• Newsletter: A collection of subscriptions that will receive messages when sent from Tiki, and an archive of sent messages.
• Message: A single edition of sent content to a newsletter, i.e. the email subject and body.
• Archive: A viewable list of messages sent to the subscribers.
• Subscriptions: The collection of email addresses to receive messages from a newsletter.
• Replace the use of "editions" with Messages Sent

Permissions

• tiki_p_subscribe_emails is to subscribe any email and tiki_p_subscribe_newsletters is to add the email related to your login. Need to clarify how these relate to the option for subscribe any email address. I guess perm override this option?
• Need to add permissions
  • tiki_p_view_newsletters_archive to view the newsletter's archived messages
  • tiki_p_admin_newsletters_subscriptions to allow for modification of the subscriptions. This is separate from tiki_p_admin_newsletters which is editing the options and permissions.
• If viewer can not subscribe or view archive, they should not see the newsletter listed.
• Make sure all permissions work.

/tiki-newsletters.php

• Shorted the URL e.g. /tiki-newsletters.php?nlId=9&info=1 e.g. /tiki-newsletters.php?id=9. This ID can then be easily linked in emailed URLs or a wiki link.
• Add a Subscribed column showing which ones the registered user is currently subscribed to.
• Admin should see
  • ”Admin” link to /tiki-admin_newsletters.php at top of page
  • Actions column like on /tiki-admin_newsletters.php
• Clicking newsletter ID, name or description should show (e.g. /tiki-newsletters.php?nlId=9&info=1) as perms allow:
  • ”List all newsletters” link back to /tiki-newsletters.php
  • Link “Send a new message” to /tiki-send_newsletters.php with this newsletter selected
  • “There are n subscriptions to this newsletter”
  • Subscribe form at top. If tiki_p_subscribe_emails or subscribe any email address is false, a registered user should see their email address as static text, not an input form. Otherwise the input form is fine.
  • Archives of message sent listed below with Subject and Date Sent. Clicking on a message will show message subject, body and sent date.
• When a user confirms (e.g. /tiki-newsletters.php?confirm_subscription=077) there needs to be a "Thank you for subscribing" shown.

/tiki-admin_newsletters.php

• Put list of news letters at top and "Create/Edit Newsetter" at bottom.
• Add anchor link for "Create New Newsletter"
• Make newsletter name bold and link to /tiki-newsletters.php?id=9

/tiki-admin_newsletter_subscriptions.php

• Page title should be: "Newsletter Name Subscriptions"
• Add check box beside each subscription and button for mass removal
• Removal confirmation should include email addresses and user names
• ”Add a subscription newsletters” should be changed to “Add a new subscription”
• Add a drop-down of all registered users, listed by username
• "Add all your site users to this newsletter (broadcast)" should be changed to “Subscribe all users from a permissions group.
• Have both forms in formcolor boxes.

/tiki-send_newsletters.php

• Sending a newsletter should not be so many steps. First form should include Data and preview button.
• Rename “Data” to “Body”
• Add help text explaining how multipart works.
• On preview two input text areas should show : one for HTML and one for plain text that was extracted from HTML. This was sender can tweak the plain text as needed.
• Where does template come from? Maybe this should be removed?
• Previews should look like a recieved email, e.g. include FROM and SUBJECT.
• "Send Newsetters" button should say "Send Message to Subscribers Now"

Upgrades aimed at 1.10

The option to allow users to subscribe any email address would define two different interfaces and logics...

Single address (default): Registered users would have to use their registered email address for subscriptions, it would not require confirmation if the address has already been confirmed through the login, but there still should be an option to verify subscription to the newsletter if the user has the option to unsubscribe.

Anonymous users could use one email address to subscribe to multiple newsletters.

Subscription would just be a check boxes beside each available newsletter.

Subscribe any email address: Registered users and anonymous have the same options except that registered users have their email address automatically inserted in the input field as default.

Subscription is an individual form for each newsletter.

There should also be an option to subscribe Tiki Permission Groups as a group, i.e who ever has that group permissions will be subscribed, and new members will be automatically subscribed.

Add search of newsletter archives.

On /tiki-newsletters.php show the same list details as on /tiki-admin_newsletters.php, except replace Action column with a Subscribed column showing which ones the registered user is currently subscribed to. Also make Name bold and break each listing into two rows, with ID, name and description on top row. Just show confirmed addresses.

On /tiki-admin_newsletter_subscriptions.php

• List what group subscription is apart of
• Add option to remove all users who are part of a selected group.

Wishlist

• News letter content was a wiki structure, that way the text is archived, but can still be revised/updated. History would show newsletter sending date.
• Each sending would be from a different page.
• The page name would be "$NewsletterName - $DateSent" and part of a structure called $NewsletterName
• HTML or Plaintext subscription options per user account i.e. not per newsletter.
• If a message is sent to multiple lists, they will only receive one copy of the email and not as many as the number of lists they are subscribed to.
• Full and automatic integration of content from the CMS (articles, comments, forum posts, blog posts, etc.) into newsletters via include statements
• Moderation option for all content, such that an admin can choose what pieces to send out via email.
• VERP (variable envelope return processing) to ensure good bounce handling
• Tracking of bounces (with bounce details) in user accounts (good for tracking mail delivery problems)
• Date tracking of when people subscribe and unsubscribe (useful for when someone leaves an account without unsubscribing, and a new person gets it and complains because they're receiving mail they didn't request)(This could be avoided if system email is used)
• Ability to track the number of e-mails opened
• Clickthrough tracking
• Capability to send html + text, with the user choosing which they receive on a global and per list basis
• custom field & personalized email capability

Outside Mailing List Integration

There has beed discussion about partnering with phplist to replace the current newsletter feature.

phplist was approched after careful analysis of all the PHP open source newsletter apps. phplist is a mature & stable with 40+ releases over than last 3.5 years. phplist has a great number of features. Michiel Dethmers, the PHPlist developer has discussed collaborating on a "connecting class" between the two.
But PHPlist is for broadcast only and the source is GPL, which can cause some issues about integration.

There has also been discussion about integration with a two mailing list system like Sympa or Mailman. But no concrete work has gone in this direction, yet.

Trackers

Bugs (SF links not working below?)

• {SF(aid=>802223)}{SF} This may be related to the evolution of newsletter subscription (which I don't know!) after the fix of 750708 - chealer.

RFEs

• {SF(aid=>749179,tag=>rfe)}{SF}
• {SF(aid=>816328,tag=>rfe)}{SF}
• {SF(aid=>654158,tag=>rfe)}{SF} (using RSS as a basis)
• Thread about newsletter feature

CVS Doc section

New for 1.8:
The following options can now be set on a per newsletter basis:

Users can subscribe/unsubscribe to this list

Determines whether users can subscribe themselves

Users can subscribe any email address

Determines whether the user can subscribe alternate addresses than the one they registered with

Add unsubscribe instructions to each newsletter

Determines whether to include an unsubscribe msg when a newsletter is sent

Validate email addresses

Determines whether the address must be validated with a test email