History: Tiki Suite brainstorming

Backup-FileSync

2014-08-04 For now, users will access and edit files via VPN, taking advantage of ClearOS's Flexshare feature. In the future, we will add a method to have offline copies of the files. Current idea is to use Syncthing along with adding ClearOS Flexshares as a storage option for the Tiki File Gallery.

• See: files vs web debate
• Because this is so true: http://xkcd.com/949/
• http://libreplanet.org/wiki/Group:SyncReplacement
• Behind the scenes folder syncing
• Mount as a local drive
• Use LDAP like Seafile
• Keep version history
• ACLs
• Check ClearOS file management with Flexshare
• http://dev.tiki.org/Web+FTP+client
• http://dev.tiki.org/File+Gallery+Revamp
• Access is important, but how important is sync? (Maybe work on remote desktop access instead?)
• Windows-Mac-Linux at least, and ideally Android as well
  • Most things on Android already sync (contacts, e-mails, etc), but automatic backups of the taken pictures/videos would be good. Perhaps sync of music as well.
  • Backup SMS and call logs to IMAP
  • And Bitcoins
  • https://f-droid.org/wiki/page/dk.jens.backup
• Encryption
• http://www.boxbackup.org/wiki/BoxComparison
• Ideally, tool helps users see what is out of sync (FreeFileSync is great at this)
• Ideally, the same tool for sync & backup. Backup being a read / add only instance of the sync tool
• In the Tiki Suite context, we control both the server and all the clients. Thus, we can / should take advantage of that. There are tools which do a great job to get around this limitation. For example, Duplicati is great for backups to any disk, but it can't do sync.
• The main use case is a central server and everyone syncs there. But there can be use cases where P2P sync (as offered by SyncThing) is interesting.
  • A user could have data on a laptop synced between two virtual machines. They could be kept in sync locally without the network, and for the sync to the main server, the data is transferred only once.
  • Some users in an organization may need to manage large files (ex.: video) but they don't need this to be backed up to the central server.

Use cases

John has several devices

John has a desktop, a laptop, a tablet, a smart phone. He wants:
a) Files synchronized on all devices (with possible exclusion of bigger files on tablet & smart phone)
b) Secure web-access to all his files and the history
c) Access to version history doesn't need to be on each device but would be nice
d) Ideally, the capacity to share a large file with a 1-time click secret URL which hell share via IM or e-mail.
e) As John's devices are sometimes all at the same place: an offsite, automated, write-only, encrypted backup to a an online drive (like Duplicati does) with version history (a backup of the last n versions of each file or all versions since x months)
f) Nice to have: a tool to search within the content of all files

Hamed, Qiao and Susana work for 3 different companies.

The three companies are teaming up for a project so they need to collaborate on some files and since they work for different companies, they can't just use the office's shared folder/drive.

Every time someone updates a file, it should synchronize the copies of the others. All versions of the file should be for sure available on the server backup. And ideally, the last X versions would also be available on each user's computer. (Or maybe just keep local revisions and the last version from the server, so as to avoid retrieving a revision which has already been replaced by a newer version.)

Hamed, Qiao and Susana could very well work for the same company but not be happy with the shared folder/drive because it doesn't provide easy access to previous versions of the files. And thus, this solution would replace the shared folder/drive.

Potential setup

• Default use case: all files are in central FlexShare, accessible via VPN
  • A personal folder for each user. Ex.: /home/firstname.lastname/
  • A folder for each team / project (set with Flexshare permissions)
• Backups
  • Every file edit provokes a backup of the file to go to a read-only archive
  • Anyone who can see the original file can also see the backups. So everyone can edit files, but each revision is stored in a read-only mode so users can restore if need be. So if a client machine is compromised, it can't delete anything permanently.
• Offline access or multiple computers
  • For users who need offline access or to sync between computer, a multi-way auto-sync process is implemented.
  • Same principal as above. Keep every single revision in read-only mode.

Syncthing

See: Syncthing

FreeFileSync

• http://freefilesync.sourceforge.net/ (Windows, Mac and GNU/Linux)
  • http://sourceforge.net/p/freefilesync/discussion/open-discussion/thread/229696d3/
  • http://sourceforge.net/p/freefilesync/discussion/open-discussion/thread/9681ff3d/
  • http://sourceforge.net/p/freefilesync/discussion/open-discussion/thread/cf1e71bc/
• synching more than two directories is not easy

BackupPC

• http://www.clearcenter.com/support/documentation/user_guide/backuppc
• http://www.clearcenter.com/marketplace/system/BackupPC.html
• BackupPC - Fresh install of 6.5 - You don't have permission to access /backuppc on this server.
• http://www.digitalkingdom.org/rlp/tiki-index.php?page=Encrypted+Remote+Backups

OwnCloud

• http://www.clearcenter.com/support/documentation/user_guide/owncloud
• OwnCloud has clients http://owncloud.org/sync-clients/
• http://owncloud.org/six/
• http://thevarguy.com/cloud-computing-services-and-business-solutions/owncloud-adds-open-source-document-collaboration-pri
• http://www.zdnet.com/owncloud-documents-to-bring-open-document-format-editing-to-private-cloud-7000022454/
• http://www.muktware.com/2013/10/pure-open-source-open-standard-google-docs-iwork-office-365-competitor-arrives/15232

Other related software

• Cyberduck WebDAV/FTP client
• https://filezilla-project.org/
  • Feature requests:
    • 1 Click Putty Integration (SSH client integration)
    • Map FTP resource as a network drive (and "send to FTP" option in file explorer)
• Hive2Hive
• Zimbra
  • http://www.reliablenetworks.com/security/zimbra-dropbox/
  • http://www.vmwareoctopus.com/
• Pydio, formerly AjaXplorer
  • http://ajaxplorer.info/extensions/desktop-sync/ https://www.ohloh.net/p/ajaxplorer http://ajaxplorer.info/ajaxplorer-synchro-client-0-8-3/
• http://www.duplicati.com/ (Windows, Mac and GNU/Linux)
  • http://groups.google.com/group/duplicati/browse_thread/thread/5854d004e551dbab
• http://www.dirsyncpro.org/ (Windows, Mac and GNU/Linux)
• http://git-annex.branchable.com/
• http://synkron.sourceforge.net/ (Windows, Mac and GNU/Linux)
• • https://www.openhub.net/forums/10/topics/5458
• http://www.areca-backup.org/ (in Java)
• http://jfilesync.sourceforge.net/
• Bonkey The Backup Monkey
• http://www.openfiler.com/community/download/
• http://labs.bittorrent.com/experiments/sync.html (Windows, Mac and GNU/Linux)
• http://seafile.com/
  • https://f-droid.org/repository/browse/?fdfilter=seafile&fdid=com.seafile.seadroid
• https://github.com/jewel/clearskies
• http://sparkleshare.org/
  • https://f-droid.org/repository/browse/?fdcategory=Internet&fdid=org.sparkleshare.android&fdstyle=grid
• Syncany
• https://www.ohloh.net/p/linshare/
• http://retroshare.sourceforge.net/
  • https://retroshareteam.wordpress.com/2013/08/29/retroshares-web-interface/
  • http://libreplanet.org/wiki?title=Group%3ASyncReplacement&action=historysubmit&diff=30210&oldid=30197
• https://zookeeper.apache.org/
  • Does more than just files
    • Files and Folders Synchronize and Distributed Encrypted Backups are on the Wish List
• http://sourceforge.net/apps/mediawiki/fuse/index.php?title=NetworkFileSystems
• http://www.ifolder.com/
• http://voices.canonical.com/ubuntuone/?p=1241
• https://www.ohloh.net/p/eyeSync
• http://mayrhofer.eu.org/dvcs-autosync
• http://www.cis.upenn.edu/~bcpierce/unison/index.html Win / GNU/Linux / Mac / BSD
  • Android port?
• http://www.csync.org/
• http://oss.linbit.com/csync2/
  • https://www.ohloh.net/p/csync2
• https://launchpad.net/deja-dup
  • http://duplicity.nongnu.org/
    • http://duply.net/
• http://en.wikipedia.org/wiki/Comparison_of_file_synchronization_software
• This is a free and open-source P2P Social Virtual Private Network (VPN) that seamlessly networks your computer with the computers of your friends
• Synchronicity
• http://mspmentor.net/cloud-computing/dropbox-box-alternatives-too-many-file-sync-sharing-options
• http://www.freenas.org/ (FreeBSD)
• http://www.openmediavault.org/
• http://www.gluster.org/
• https://tahoe-lafs.org/
  • https://leastauthority.com
• http://tools.tortoisesvn.net/CryptSync.html
• http://ori.scs.stanford.edu/
• OpenAFS
• http://luckybackup.sourceforge.net/
• http://www.liferay.com/products/liferay-sync/features

File managers

CubicExplorer

GNU Midnight Commander

Double Commander

muCommander

Far Manager

Vifm

CMIS

• http://en.wikipedia.org/wiki/Content_Management_Interoperability_Services#CMIS_Servers
• https://www.ohloh.net/p/CmisSync/
• https://wiki.oasis-open.org/cmis/CMIS%201.0%20Products
• http://chemistry.apache.org/phpclient.html
• https://www.ohloh.net/p/CmisSync
• https://github.com/scara/moodle-repository_cmis
• http://foswiki.org/Extensions/CmisPlugin
• https://drupal.org/project/cmis

LibreOffice

LibreOffice has a lot of features, and most overlap with Tiki functionality

In general, the desktop app (ex.: LibreOffice) will tend to be more powerful, and better to produce a printable version while the web app (ex.: Tiki) will make it easier to make web documents and to have multiple users collaborate (especially multilingual collaboration). Users will have to choose on a case-by-case basis which one to use and be made aware of the tradeoffs. With the File Gallery features (tags, categories, permissions, check-in, check-out, lock and WebDAV support), we'll minimizes the annoyances of the inherent limitation of file-based computing. On the other hand, with HTML5 and AJAX, we'll strive to make the web app experience better & better.

In any case, the transversal features such as unified search, fined-grained permissions, tags, categories, category transitions (for workflows) will work be it a Tiki spreadsheet or a Libre Office spreadsheet.

Newer organizations which are very collaborative and web-based may tend to create a lot of the new documents in Tiki. On the other hand, organizations with a lot of history will have a large number of files and they'll want to simply copy over to the new shared disk.

Thus, users can edit their .ODF files in LibreOffice via WebDAV. A Tiki plugin for LibreOffice could be made to do certain operations to a given document from within LibreOffice (just to avoid that the person edits a document and also needs to go via the web interface to add tags / change categories / etc.) At the very simplest, it could be "open the URL with the metadata about this file".

The other integration is to add WebODF support. When in the File Gallery, if it's a file format supported by WebODF, a drop down menu item "edit via browser" would appear. This is exactly what we do in Tiki7 when a .svg file is in the file gallery as it is editable in the browser via SVG-edit.

So the idea is to leverage the best cross-platform ODF editor now (LibreOffice), which covers PDF needs and is a familiar paradigm, while also having a fully Web way of doing things with WebODF. Users could choose to do most of the work with LibreOffice via WebDAV and use WebODF for reading and minor edits from a mobile device.

Over time, there will likely be some semi-automated ways to convert from one format to the other. For example, you may want to have ongoing collaboration with the Wiki and Structures, but every so often, export the document to LibreOffice to take advantage of page layout, pagination and PDF export. That is what the Tiki community does to generate a printable version of the documentation

WebDAV-Cyberduck

Even if files are on the Tiki Suite server, we still want the double-click, edit and save experience of a desktop file. Via the File Server or via the File Gallery, binary files will be accessible via WebDAV. Thus, a WebDAV client is needed and Cyberduck is a nice app for Windows and Mac.

Please see: Cyberduck. License: GPL

(There is also the built in client in Mac OS's Finder, though it's not exactly as practical as Cyberduck, it is a no download alternative)

It's also nice for WebDAV access to Wiki Pages

Virtualization / Desktop as a Service / Provisioning

See also: Tiki Suite Desktop
This will be more & more important as Tiki Suite SaaS gets underway. Jitsi can do Desktop streaming and Provisioning. We also want a solution like GLPI and OCS Inventory NG

• http://en.wikipedia.org/wiki/Comparison_of_remote_desktop_software
• https://en.wikipedia.org/wiki/Desktop_virtualization
• http://www.winsupsite.com/vdi-for-free-alternatives/
• https://www.ohloh.net/tags/virtualization
• http://code.google.com/p/ganeti/
• Proxmox
• https://www.ohloh.net/p/foss-cloud
• http://www.ulteo.com/
• http://trac.zentyal.org/wiki/Documentation/Community/ZentyalDesktop/Ubuntu
• http://wiki.x2go.org/
• http://www.xrdp.org/
• http://en.wikipedia.org/wiki/Apache_CloudStack
• https://www.ohloh.net/p/xenproject-hypervisor
• https://www.ohloh.net/p/noVNC
• http://guac-dev.org/
• Xen
• LXC::
• http://cantivo.org/
• https://www.ohloh.net/p/virt-manager
• OpenStack
• https://code.google.com/p/ovz-web-panel/
• http://classic.slashdot.org/story/13/11/06/1743216

ClearOS feature request

This is a draft. When ready, will be posted to http://tracker.clearfoundation.com/

This permits to add Linux or Windows VMs:
http://www.clearcenter.com/support/documentation/clearos_guides/install_virtualization_using_kvm
http://www.clearcenter.com/support/documentation/clearos_guides/install_virtualization_using_virtualbox

The next step is to have an app, with a GUI like Kimchi or WebVirtMgr

• For the admin
  • A web GUI to create new VMs and assign permissions, activate RDP
  • LDAP integration (manage which use has access to which VMs)
  • Port forwarding & firewall
  • Make snapshots
  • Setup automated backups
  • Setup file storage (Flexshares)
• For the end user
  • Access via RDP and VNC / Spice / etc.
  • Roaming profiles
  • Access to the FlexShares to which they allowed

Use cases:

• Running a Windows server on top of ClearOS
• Running some Windows clients for dedicated apps on top of ClearOS
  • An organization may have a limited number of licenses of expensive apps (AutoCAD, Photoshop, etc) which some team members only need infrequently or that several users need access to but the app is not designed to be multi-user (ex.: legacy Accounting software)
• Team members with their complete desktop on VMs hosted by the ClearOS server and only access via RDP / Spice / VNC
  • Benefits: avoid support of configuring their PC

KVM

• KVM
• http://www.linux-kvm.org/page/Management_Tools
• http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine#Graphical_management_tools
• http://www.clearcenter.com/support/documentation/clearos_guides/install_virtualization_using_kvm0

http://lists.ovirt.org/pipermail/board/2013-July/000912.html
http://en.wikipedia.org/wiki/KIMCHI#Kimchi.E2.80.99s_Role_in_the_KVM_Ecosystem
http://www.ovirt.org/How_to_create_a_Windows_7_Virtual_Machine

VirtualBox

• http://sourceforge.net/projects/phpvirtualbox/
• http://www.ubuntugeek.com/vrdp-virtualbox-remote-desktop-protocol.html
• http://www.clearcenter.com/support/documentation/clearos_guides/install_virtualization_using_virtualbox
  • http://sourceforge.net/p/phpvirtualbox/wiki/Authentication%20Modules/#ldap
  • http://www.clearfoundation.com/component/option,com_kunena/Itemid,232/catid,40/func,view/id,2834/
• http://www.clearcenter.com/support/documentation/clearos_guides/install_virtualization_using_virtualbox

Password management

• Let's decide first about files vs web and come back to this.
  • Or maybe both 😊 https://github.com/shkdee/KeePassPHP See KeePass
• Perhaps we should also tackle bookmarks with the Firefox Sync Server ?
• https://wiki.koumbit.net/action/show/PasswordManagementService/SoftwareComparison
• http://www.infoworld.com/d/security/review-the-best-password-managers-pcs-macs-and-mobile-devices-244519?page=0,5

Print

• Online Publishing House

Bitcoin

• We want a way to make and receive payments via Bitcoin

IRC

• Main focus is XMPP chatrooms, but bridge from IRC to XMPP would be nice
• https://www.ohloh.net/p/quassel
• https://www.ohloh.net/p/seamonkey
• Jitsi will eventually support IRC
  • https://github.com/jitsi/jitsi/pull/16
  • https://www.mail-archive.com/dev@jitsi.org/msg03672.html

Building Management, Physical security, etc

• Remote surveillance / alarm system
• http://serverfault.com/questions/342380/open-source-system-for-swipe-card-access
• http://en.wikipedia.org/wiki/Building_management_system

Software development

• Source Control Management
• Integrated development environment
• Tiki Suite Development

To be decided

• Should SeaMonkey replace Firefox & Thunderbird?
  • Or Claws Mail via Gpg4win
    • See also: https://gpgtools.org/gpgsuite.html
• F2F
• An SEO tool
  • Mostly useful for web agency use case
• StatusNet Superceded by pump.io Status updates/micro-blogging (distributed) or wish2258
  • cdrwhite: how about a few minimal extensions to the shoutbox stuff, we already have? shouts-per-user (also as rss), twitter-account and facebook-account as objects with object permissions, allowing a group to use one account)
    • marclaporte: Yes, that is an option, and making interoperable with StatusNet
• Business Intelligence
• How TRIM fits in
  • Hosting companies could offer this as SAAS: http://tiki.org/SWOT#SaaS_Hosted_solutions_Wikifarm_D_ and maintain Tiki instances with TRIM
  • More details at: http://tiki.org/Business+Models#Advanced_hosting
• High-volume newsletters
  • Tiki newletters are fine for small volumes, but they need to be improved for large volumes and the option to use an external service would be nice (because of spam protection)
• Remote provisioning like Jitsi does it.
• Tiki Suite Health

See also

• https://lorea.org/
• https://getsyme.com
• http://p2pfoundation.net/Category:P2P_Infrastructure
• http://p2pfoundation.net/Category:Technology

Old brainstorming