Developer Discussion: Fabrik's internal approach

[Sophist]

I don't want anyone to get the wrong idea about this post. I am a fan of fabrik and believe in the overall design (for example the use of plugins is brilliant)...

However, I can't help but feel that the number of bugs we are seeing and reporting are a product of the way some of the internals have been implemented which seem far more complicated than it needs to be. So I would like to open up a discussion on whether other people agree with my thoughts and about whether it makes sense to simplify and clean before 3.1 is released. So, here are my thoughts...

1. At present we have Lists, Forms, Groups and Elements. But it seems to me that Lists are actually doing two things: A. They define the display of data in a summary tabular form; and B. They define the structure of the database i.e. which tables join to which. I am not sure whether this makes sense or whether these two functions should be separated.

2. I think that database functions should be separated out into a handler - by doing this you can have the Lists, Forms, Groups & Elements provided with just the data they want and leave it to the database handler to determine the best way of fetching this data.

3. The way that Repeating Groups are handled seems to me to be over complicated.

At present for a list we either do a single select and receive multiple rows (because of the join with the repeating group) which are displayed even if the user doesn't want any of the repeating data displayed), or if you set Merge Rows in the List, then it makes multiple Select calls, with second and further calls based on the individual ids from the master table which for a large table could potentially be quite bad for performance.

However it seems to me that (in most cases at least) Lists will not display repeating groups - and if the developer wants to do this, then we really need a tree-based display of data in Fabrik to handle this (which is a separate discussion topic really). So lists will mostly only need data from the main table, in which case we can make a single Select which returns the exact data we need and avoid doing a join which returns multiple rows for each row in the master table which then need to be merged.

On the other hand, a Form only returns one row for the master table, and we could then do a second Select for the repeating groups i.e. a maximum of two calls, each of which will be against a single key.

4. It seems to me that with a brand new release for Joomla3, we have an opportunity to clean up the structure and code and make it simpler and hence less buggy.

So, what do other people think...??

P

 

[troester]

I don't want to join a structural discussion (leaving this to Rob and Hugh) but I don't agree with this one

However it seems to me that (in most cases at least) Lists will not display repeating groups

At least in my case if I have a list with joins I usually want to display (some of) the joined data in list view.
If you want to display only your main DB table you can always create an additional list without any join pointing to this table (e.g. by copying your first list + deleting the joins)

 

[Sophist]

Troester

I have tried this and I now see what you mean - when you select merge or merge and reduce, you get the repeating data provided as a unordered list.

P

 

[Bauer]

You're starting to sound like me when I first started participating in the discussions, Protapia. I've come to the conclusion that its best not to ruffle feathers or try to make fabrik into your vision. I can't think of another Joomla extension I have ever used that is more complex (with perhaps the exception of feedgator) - so its understandable why the developers are in no hurry to re-invent the wheel.

However, that said, I wholeheartedly agree that there is plenty of bloat - and most of all inconsistency in the way things are being done.

I'd like to see fabrik go by the same rules as Joomla - wherever and whenever they possibly can. So to respond to each talking point in your post...

1. I'm fine with the database interaction - just not in the way it is being done. Like I said, consistency is by biggest beef. There are mysql functions for retrieving anything you need to know about the table structure - and IMO those should be used to create any fabrik table wherever possible, rather than making the user set the properties. This goes for 'internal id' size - text widths - integer or decimal specs - ALL that is already available by querying the tab structure. The same is true for table properties such as NULL - Allow null value - and setting the default values used (as set in fabrik with Eval), non-integer primary indexes, compound (concat) indexes, etc. All of that is already set in the database table schema when fabrik creates a list from the table, no?

2. Not sure what you're saying here. But I would like the ability to create and use temporary (user-defined) queries that in turn could update related tables. Have a look at the way tablegear works for an example of how this CAN be done. (I'd really love to see someone pick up on that (abandoned?) project and make a Joomla extension from it - or as a plugin for fabrik.)

3. I couldn't agree more. Though I'm sure there's a historical (or as we used to say in my programming days- 'hysterical') reason for it. The repeat tables that get created add real bloat and overhead to the server with all the queries that need to be made - when in most instances a simple JSON object could be stored as the value for the element - in the same table - and would do the same thing.

4. Like getting rid of old plugins that have overlapping features of others? (Such at date and timestamp). And most of all, make all elements play by the same rules - and have the same option included (like read-only, hide). I understand that many of these plugins were borrowed from other sources and integrated into fabrik - but still they need to 'meet the specification' of a fabrik element - or be modified - for the sake of consistency.

5. (Can I add a #5?) Focus more on fixing the long-standing bugs and issues - rather than the 'take them as they come' approach that caters only to 'paying' customers' individual needs. There are bugs that still exits today that are unchanged from a year ago. I just think more focus needs to be placed on getting the inner workings of the extension working - as perfect as possible - before even more bloat is added - like plugins that few people will ever really use. If those bugs were fixed, (most notably, with the databasejoin plugin) I too would become a paying customer. But like I explained long ago, I am not going to pay to do what I am doing now - which is, 99.9% of the time, just reporting bugs.

If you measure the value of an extension by it's size, then you've found your goldmine. But, personally, I'd love to see fabrik become lean and mean instead of fat and lazy - and getting fatter by the day, it seems.

 

[Sophist]

I deliberately started my OP with a statement about how I am a fan of Fabrik - as you say it is a utility with very broad functionality and ability to develop complex solutions, and the owners make it available free to the likes of me (and you?) so good on them.

They key point of my OP was that I have a feeling that the "bloat" (as you put it) - or what I might call the historical divergence from architectural purity (which is a problem that many many development projects face - not just Fabrik) - is the cause of a lot of bugs. Often this sort of divergence is due to an accumulation of previous bug-fixes which solved the immediate bug at the expense of purity - and without this purity, fixes to today's bugs are probably only going to make this worse.

I think that Rob and Hugh have their work cut out supporting the non-beta versions(s) and by themselves I think they are struggling to cope with the number of posts to be read, bugs to be fixed, dealing with the web-site spam, and all the other work items that come from being the central team.

I cannot find a license document on the web site, though their subscription page says that it is an open source project under a GPL2 license. But whilst in theory someone could fork the code and develop it themselves as a competing project, that is not something I would want to see happen. I certainly have no desire or intention of doing this, and feel that the best route is for this to continue under Rob and Hugh's stewardship.

Personally, I want to give something back to say thank you for the free software - which is why I try to fix whatever bugs I can and push the fixes back to GitHub for inclusion - and I wonder if Rob and Hugh are not missing a trick here by harnessing this sort of user capability. So - and I admit I am adding to the debate here and hope that Rob / Hugh will take this in the spirit of helpful idea generation that is intended - here are a few more possibilities to throw into the pot.

1. Fabrikar should try to harness the community more by publishing an issues list and allowing the community members to pick issues to fix. Issues could be code, wiki, etc.

2. They could host two weekly chats (on IRC??) - one for user help and support and one for Fabrik development. For the user help and support chat, I wouldn't be surprised if they got lots of expert users offering to provide support, so it might not be much of a workload for them - and for a limited investment of time they might gain an even better insight into the issues the user base are facing. For the development chat, this would enable them to engage with the community for prioritising issue fixing and to get input to help them with setting strategic development directions.

I can understand why Rob and Hugh focus on what they focus on and why this leaves bugs unfixed for lengthy periods (so this is not a criticism), and this suggestion is a way of reducing the pressures on their time, helping the product quality and functionality along, get fixes done faster, result in better overall support and so have a happier user base and allow them to focus on value-added and income-generating activities such as core-web-site support, strategic direction setting and development and paid support / bespoke enhancements. I would imagine that they might feel that better community support might take away some of their paid income, however I suspect that this might be more than compensated for by growth in usage.

3. Finally, it might be nice to recognise those community members who contribute to the product development and support by providing a free upgrade to a higher level of membership - and though clearly I have a personal interest in this, I would expect to see some sort of longevity requirements which I do not yet meet. Again, I suspect that they might gain more from e.g. members providing support to other members in forums they currently do not have access to than they will lose in income.

So, as I said originally, this post is here to stimulate some discussion and idea generation, and perhaps to help foster an even more engaged community.

P

 

[Sophist]

P.S. Whilst I applaud Fabrikar for making available for free what previously required a Supporter subscription, I am not clear whether the supporter subscription still exists and if so what benefits you get from it. One suggestion would be to allow Supporters to reply to existing threads in the Pro / Standard forums, but NOT to start new threads. In simple terms, ?39 pa is not much to pay for even relatively minor benefits.

P.P.S. Whilst the existing Wiki stuff is good (or at least it will be once the links are fixed) it is IMO somewhat ad-hoc rather than comprehensive. One idea for community involvement would be to create more comprehensive documentation from high-level to detail, including every plugin, every setting, every css class etc. This is a big job that really needs to be crowd-sourced.

 

[Bauer]

You do realize we're turning this thread into a soap opera?

I feel no envy for any Joomla developer. With the way the core API code changes every few years I?d find it impossible just to keep up with making it all work with the Joomla core.

I agree with all your observations. I?ll give you another example. The feedgator project has been in limbo for over a year now. Matt has gotten so behind that it was pulled from the extensions at Joomla.org. But nothing comes close to doing what that extension could do with rss feeds. It?s a pity that project seems to be stagnant at the moment -and most all of the issues are simply because it got too big to handle - and relied on a lot of other 3rd party libraries that kept changing on him.

When multiple hands are feeding the same beast and the left hand doesn't know what the right hand is doing, sooner or later someone is bound to get a hand bitten off. I think that is what you were saying - though without my usual analogical snark.

I?m into the fabrik project now ? up to my neck in it - and there?s no turning back. The last thing I need is to p-off the developers to the point that they just give up.

And I?m on the same page with you when it comes to my willingness to show my appreciation in any way I can. I?m unemployed and disabled for almost 20 years now, so spending money on software is not something I can take lightly. But that said, there have been many a time where I am so frustrated, I lose my cool (most of that just one of the many symptoms of my disability that I?ve been living with). Also, I worked fulltime as a programmer for a decade or so ? and know all the headaches that come along with it ? including dealing with customers (aka ?stupid users?).

On to your latest itemized points?
1. They really already have that ? though maybe now that they have a nicer/newer version they will have better luck with it. I plead guilty on this one. I wish I had the time to contribute more and become involved ? but I just don?t.

2.That would be cool. But again, I have my ?issues? that would keep me out of the loop of something like that. I used to be a social butterfly ? I?d consider myself a ?pioneer? on the net in that regard - yet today I don?t even twitter, tweet, etc. And I understand the ?business? aspects. Someone?s gotta pay the bills. I don?t mean to sound demanding, only helpfully suggestive ? though I know that many times I don?t come across like that.

3. I hear ya. Though at the moment, there is no need - and I?ll cross that bridge when I come to it. I have no idea how much this project ?brings in? to the folks at fabrikar.com, but my guess is not very much. I?d like to think that any ?word-of-mouth? recommendations and ?advertising? I do for fabrik more than makes up for my inability to contribute otherwise. At the same time, I?ve been ?working? on this short-term project that has turned into more than a year?s work, and would calculate my ?pay? at about $2-$3 an hour (really). I?m doing this more for the love of my work ? or a ?hobby?. But it already takes up far too much of my ever-dwindling life. I realize that not everyone has that attitude or outlook on life ? especially younger people who are always working with visions of dollar signs spinning in their heads. Yeah that would all be nice ? but I?m way past thinking I?ll ever get rich sitting behind a PC. So more power and wealth to anyone who does.

 

[rob]

1. At present we have Lists, Forms, Groups and Elements. But it seems to me that Lists are actually doing two things: A. They define the display of data in a summary tabular form; and B. They define the structure of the database i.e. which tables join to which. I am not sure whether this makes sense or whether these two functions should be separated.

What would be the benefit of splitting them out? Im all for making that list.php class smaller more modualized but we would need to do it in a way that was safe and logical. Currently I haven't seen a right way to implement that. In 3.1 a lot of the db insert logic has been delegated to groups/elements which makes the form process code a lot more readable.

2. I think that database functions should be separated out into a handler - by doing this you can have the Lists, Forms, Groups & Elements provided with just the data they want and leave it to the database handler to determine the best way of fetching this data.

I'm not sure that I understand what you are describing here. In general I'd find it hard to see how you can separate them further than they are.

3. The way that Repeating Groups are handled seems to me to be over complicated.

At present for a list we either do a single select and receive multiple rows ....with second and further calls based on the individual ids from the master table which for a large table could potentially be quite bad for performance.

Not sure that its that bad for performance. As you are selecting on primary key values the queries are very fast, so even doing 2 is really not that much of a time issue (compared to other areas in Joomla/fabrik for instance) Really this is more the way that relational (as opposed to noSQL database work). The only way round that might be to just load the main list and then load related data via drill-down ajax calls, in a similar manner to the way the related data sections work now.

However it seems to me that (in most cases at least) Lists will not display repeating groups

Correct but I think in this case when there are no joins and merge data is not on then only a single query is run

....- and if the developer wants to do this, then we really need a tree-based display of data in Fabrik ....

agree - see my comment above.

On the other hand, a Form only returns one row for the master table, and we could then do a second Select for the repeating groups i.e. a maximum of two calls, each of which will be against a single key.

I'm not sure what the advantage of this would be over the way we currently do things?

4. It seems to me that with a brand new release for Joomla3, we have an opportunity to clean up the structure and code and make it simpler and hence less buggy.

All major changes to the database management are really going realistically set back a release date by at least 9 months. There are massive combinations of things to test and we are still working through the changes made for the repeat elements. I'm not in the mind to really start messing any more with the core, within Fabrik, its simply not a cost effective task for us to be doing.

I've come to the conclusion that its best not to ruffle feathers or try to make fabrik into your vision. .... so its understandable why the developers are in no hurry to re-invent the wheel.

Well I'd say ruffling feathers is fine, but making fabrik into anyones vision other than our own is going to be hard. That sounds dictatorial but the fact is that in 4 years time we will still be here living with the decisions made today. We already do live with those decissions for better or for worse, a prime example being the date element's 'save as local time' option. This was something someone needed, so we did it, but in the end it was only pretty much for that one person and yet that option has probably resulted in days of debugging, fixing, bugs, general confusion and basically unwieldy code. Unfortunately once 'in' its very hard to remove features and still offer an upgrade path from say 3.0 to 3.1.

Focus more on fixing the long-standing bugs and issues - rather than the 'take them as they come' approach that caters only to 'paying' customers' individual needs

I think we try to balance this, if there are important outstanding, reported, with case example bugs that we can fix that we deem important we do so. However, we try (and I really do only mean try) to make a living out of Fabrik so paying customers come first. Otherwise we'd have no money and no time to devote to fabrik at all.

If those bugs were fixed, (most notably, with the databasejoin plugin) I too would become a paying customer. But like I explained long ago, I am not going to pay to do what I am doing now - which is, 99.9% of the time, just reporting bugs.

Ok so spend 5% of the time reporting bugs and 95% of the time fixing them - I'm not sure which dbjoin bugs you are referring to so I can't comment on the issue you are referring to.

"bloat" (as you put it) - or what I might call the historical divergence from architectural purity

Yes there is some of that, which is why you'll see me really not be for adding more options. I would say that the core is not where the majority of bugs are. It is in plugins and then the mis-use of said plugins (e.g. the autofill form plugin which I wish I had never released, as whilst its very powerful, it produces a lot of support queries with people mis-using it)
There's also the fact that this project started eight years ago, and we've learnt a lot in that time, so somethings are done well and others not so well. The iterations over those eight years, we have gone through really have helped specifically with the core features.

1. Fabrikar should try to harness the community more by publishing an issues list and allowing the community members to pick issues to fix. Issues could be code, wiki, etc.

There's the bug list on github, we may try to implement that back into the forum / website somehow, but again that's more work for us and I think the people who are willing / able to contribute are already doing so. There was a wiki page saying what needed to be done, and with loads of placeholders saying 'coming soon' etc, but basically things never came, and the to-do page was never used. If someone wants to add to the wiki they can do so now. There's nothing stopping them. We have a pretty good flow of translations at transifex.com going which was set up recently.

They could host two weekly chats ....

Chat support is extremely time consuming and not at all for the faint hearted. E.g. today I spent 5 hours with one client. You have to understand the project they are working on,communicating often in non native languages, then understand what they have implemented (which is often not at all how you would have done things). I don't think we would find people willing to submit to that type of help for free and then we would need to worry about the quality of the support/advice given.

- and for a limited investment of time they might gain an even better insight into the issues the user base are facing.

To be honest we eat breath fabrik every working day, helping users to implement things and explaining things, I'm not sure an additional chat method would really help us gain further insight.

I am not clear whether the supporter subscription still exists and if so what benefits you get from it. ...allow Supporters to reply to existing threads in the Pro / Standard forums, but NOT to start new threads.

Supporters now have access to downloads that free users don't. I would be against allowing supporters to reply to pro/standard forums. Whilst that sounds like a good idea for improving community support, it would lead to thread hijacking where non-paying users basically ask 'me too' questions which are not at all related. We have been down that route and that is what happened, and its tiring for us to continually repeat to people not to do it and to have to trim forum posts etc.

One idea for community involvement would be to create more comprehensive documentation from high-level to detail, including every plugin, every setting, every css class etc. This is a big job that really needs to be crowd-sourced.

I guess we could write a blog post to ask people to help. I'm kind of jaded about doing these types of requests though. Experience is that if people want to help they do, but 99% of people are consumers and don't want to get involved, or feel that the project is too complex to be able to provide useful documentation.

I feel no envy for any Joomla developer. With the way the core API code changes every few years I?d find it impossible just to keep up with making it all work with the Joomla core.

Amen, this is a major source of stagnation within what we do.

I have no idea how much this project ?brings in? to the folks at fabrikar.com, but my guess is not very much.

I'm toying with the idea of publishing the figures. It might be interesting for people to see?

I?d like to think that any ?word-of-mouth? recommendations and ?advertising? I do for fabrik more than makes up for my inability to contribute otherwise.

Word of mouth is priceless thanks for spreading the word!

Finally I'd say that we are playing with ideas for Fabrik4 which will be a radical departure from Fabrik3.x. But all of that is purely at a R&D phase. my random thoughts are being compiled here https://github.com/Fabrik/fabrik/wiki/future-json-structure

 

[Sophist]

Rob

I want to thank you for taking the time away from other activities to write such an extended response. My reply is equally long so I am going to split it into a couple of sections, the first part looking at technical bits.

What would be the benefit of splitting them - the table definitions and list displays - out?

I can imagine the need for having several different lists off the same tables, so each table could have several Groups, each Group could have several elements, each list/form could have several groups which could relate to several tables in which case joins would be needed. The groups would be able to tell the table which fields from the tables would be needed for the elements, and only those elements needed would be fetched from the tables.

I'm not sure that I understand what you are describing here. In general I'd find it hard to see how you can separate them further than they are.

I am imagining the normal benefits of modularisation i.e. separating out the "what data is needed" request from the list/group etc. from the "how do we best get it" and centralising the DB handling into a single place might allow you to make improvements to DB handling across the whole of Fabrik with a change made once in one place. So we separate the what (data is needed) from the how (do we access it).

Not sure that its that bad for performance. As you are selecting on primary key values the queries are very fast, so even doing 2 is really not that much of a time issue (compared to other areas in Joomla/fabrik for instance) Really this is more the way that relational (as opposed to noSQL database work).

I guess so long as it is not one request plus another one per row, several is OK. I guess this is a trade-off between multiple calls to get just the amount of data you want and a single call returning e.g. multiple copies of the main table rows which then need to be consolidated inside php. You would know best which this is. (Same applies to getting each value twice from a select (raw and non-raw) ? again you will know best what the performance tradeoffs are.)

The only way round that might be to just load the main list and then load related data via drill-down ajax calls, in a similar manner to the way the related data sections work now.

I am not sure that you want to rely on Ajax calls for basic functionality - some people turn off js for security reasons, so ideally (and it may not be possible with something as complex as Fabrik) the site would still work (all-be-it without the fancy benefits of JS).

Correct but I think in this case when there are no joins and merge data is not on then only a single query is run.

I don't display my repeating data in my list, but it still does either a single joined Select and shows repeated lines or does multiple (3 or more) calls depending on the setting in the List/Data/Joins/Display mode. This is the sort of area where it seems to me that if the List/Table were separated, the List would ask the Table for certain data, then the Table would intelligently do the join only if needed and return the data in the way it is needed.

All major changes to the database management are really going realistically set back a release date by at least 9 months. There are massive combinations of things to test and we are still working through the changes made for the repeat elements. I'm not in the mind to really start messing any more with the core, within Fabrik, its simply not a cost effective task for us to be doing.

It is always a difficult decision where and whether to make significant structural changes which will inevitably bring in new bugs and need lots of fixes. I agree totally that you shouldn't introduce e.g. restructured database operations and set back the release date.

It is also a difficult decision whether to maintain a seamless upgrade path for all users (which means keeping all the problem legacy code) or to provide a seamless upgrade for users who have only used basic functionality and require some rework of configuration, js, css, eval'd fields, bespoke templates etc. for more advanced users.

Joomla generally seem to provide transition approaches i.e. in v2.4 you did it the old way, in v2.5/2.6 you can do it either the old or the new way, in v3 you can only do it the new way. You are already considering that F4 might be a departure from F3, and I would be happy to contribute to some thinking about how F4 might be different and see how the user impacting bits could be backward ported into F31.

Well I'd say ruffling feathers is fine, but making fabrik into anyones vision other than our own is going to be hard. That sounds dictatorial but the fact is that in 4 years time we will still be here living with the decisions made today.

Couldn't agree more - there has to be someone making the final decisions which we will have to live with and that has to be you. However, I sense that some users feel that they would like to contribute design ideas (which you would ultimately decide upon), and that they find this difficult.

We already do live with those decisions for better or for worse, a prime example being the date element's 'save as local time' option. ... Unfortunately once 'in' its very hard to remove features and still offer an upgrade path from say 3.0 to 3.1.

Exactly.

Yes there is some of that, which is why you'll see me really not be for adding more options. I would say that the core is not where the majority of bugs are. It is in plugins and then the mis-use of said plugins (e.g. the autofill form plugin which I wish I had never released, as whilst its very powerful, it produces a lot of support queries with people mis-using it).

I am not sure I would agree with this - in my short experience with Fabrik (which is the best development extension for Joomla by far) I am not using many plugins but have found numerous bugs and inconsistencies which are almost certainly core code. (I am including e.g. the field element plugin as core code here.) So e.g. off the top of my head:

• textarea fields not populating the non-raw placeholders
• repeating groups not working on multi-page forms
• same list or form in front-end not working same as in back-end (e.g. some fields populating in one but not the other)

There's also the fact that this project started eight years ago, and we've learnt a lot in that time, so somethings are done well and others not so well. The iterations over those eight years, we have gone through really have helped specifically with the core features.

As above, choosing when to stick with a mistake and provide compatibility and when to fix it and require user to rework is a tricky decision. But the consequences of leaving code which (with hindsight) is poorly architected / implemented can be substantial i.e. inconsistent results depending on minor settings here or there which are the devil to debug (difficult to track down, difficult to understand the existing code, difficult to decide how to fix it) and which are likely to require equally specific fixes which just make the overall code quality worse.

Amen, this (frequently changing Joomla structure) is a major source of stagnation within what we do.

Joomla make such changes for a reason, despite the knowledge that they make backward compatibility difficult. Essentially they know that they got it wrong at an earlier point and need to make a design change in order to preserve architectural purity moving forward. So they bit the bullet and did it. It seems to me that this is the stage that Fabrik is at ? needing to clean up in order to stabilise for the future.

Finally I'd say that we are playing with ideas for Fabrik4 which will be a radical departure from Fabrik3.x. But all of that is purely at a R&D phase. my random thoughts are being compiled here https://github.com/Fabrik/fabrik/wiki/future-json-structure

This could be an opportunity to expand this into an idea collecting base from knowledgeable users.

P

 

[Sophist]

Rob

Thanks again for taking the time away from other activities to write your extended response. Here is the second part of my reply looking at community involvement.

I think we try to balance this, if there are important outstanding, reported, with case example bugs that we can fix that we deem important we do so. However, we try (and I really do only mean try) to make a living out of Fabrik so paying customers come first. Otherwise we'd have no money and no time to devote to fabrik at all.

...
Ok so spend 5% of the time reporting bugs and 95% of the time fixing them

- I'm not sure which dbjoin bugs you are referring to so I can't comment on the issue you are referring to.

I agree - if you have the technical skills to diagnose some or all of the cause of a problem, you probably have the technical skills to identify the fix - in which case you should learn how to use GitHub so that the everyone can benefit from the effort that you have already put in.

That said, the code can be damn hard to understand - not much detailed documentation, and where there are extensive comments, it is often to explain why a kludge has been implemented.

There's the bug list on github, we may try to implement that back into the forum / website somehow, but again that's more work for us and I think the people who are willing / able to contribute are already doing so.

I think that anyone who is going to contribute should be willing to get up to speed on GitHub. The Issues list on GitHub is in theory a good place to provide a To-Do list, however it gets polluted by automatic documentation of PRs (which makes it difficult to see what still needs doing), and you cannot assign an issue to yourself to say you will fix it. Another project I contribute to uses Jira for bug tracking - so I don't know if that would be an easy alternative.

There was a wiki page saying what needed to be done, and with loads of placeholders saying 'coming soon' etc, but basically things never came, and the to-do page was never used. If someone wants to add to the wiki they can do so now. There's nothing stopping them. We have a pretty good flow of translations at transifex.com going which was set up recently.

I think we need a little more coordination if we are to get better contributions from the community. We need clear To-Do lists with an ability to prioritise, ability for people to volunteer to take ownership of an item and progress it etc. As volunteers rather than employees they need to be allowed to choose what and how much to do, but the community needs a steer from you as to what is important and you need to trust that they want to see Fabrik succeed as much as you do and work (at least most of the time) on stuff which is important as well as stuff which impacts them.

Chat support is extremely time consuming and not at all for the faint hearted. ... I don't think we would find people willing to submit to that type of help for free and then we would need to worry about the quality of the support/advice given. To be honest we eat breath fabrik every working day, helping users to implement things and explaining things, I'm not sure an additional chat method would really help us gain further insight.

If they need that level of support, then they do need to pay you to provide it. But a lot of what I read on the forums are from people who need help rather than support where you don't need to provide that level of depth.

The chat's I was thinking about are more about engagement with the non-paying community than about support. The paying community is (I imagine) very different to the non-paying community. I imagine that paying users are expecting you to diagnose and fix things, so chats will be in depth about their problem and therefore time-consuming. I think the idea is to encourage some of the non-paying community to fix things for themselves and other non-paying users (and in doing so take some of that workload off you so you can concentrate your non-paid time on higher value stuff), but they need both a steer about where you need help and a sense that they are being listened to.

Supporters now have access to downloads that free users don't.

I am unclear what these downloads are. But if I felt they were valuable, I would pay.

I would be against allowing supporters to reply to pro/standard forums. Whilst that sounds like a good idea for improving community support, it would lead to thread hijacking where non-paying users basically ask 'me too' questions which are not at all related. We have been down that route and that is what happened, and its tiring for us to continually repeat to people not to do it and to have to trim forum posts etc.

Yes ? I can see that. But I feel you are missing a trick as there are some users who will not abuse this and who could help you. So perhaps this could be an equivalent to the Microsoft Valued Professional tag i.e. given to people who are giving something back on a discretionary basis.

I guess we could write a blog post to ask people to help. I'm kind of jaded about doing these types of requests though. Experience is that if people want to help they do, but 99% of people are consumers and don't want to get involved, or feel that the project is too complex to be able to provide useful documentation.

I think that managing communities can be both hard and frustrating. Yes, 99% of people are relatively non-technical consumers who want to make a simple list and that's it. They don't have the skills and probably won't accumulate the knowledge to be able to contribute. And IMO that's fine.

But the other 1% of users who want to do more, who go through the learning curve, who appreciate that they have had the benefit of the free software and want (or could be encouraged) to give something back can be of great benefit to you - but they need:

1. To be encouraged to recognise the benefit they have had and to give something back.
2. To have it made easy for them to contribute - by improving documentation, by doing testing, by debugging and fixing - whatever their skills and time allows. If it is too difficult, then they won't bother.

I'm toying with the idea of publishing the figures. It might be interesting for people to see?

I am not sure that we need to pry into your personal finances. Since you made this an open-source project and spend a lot of time on un-paid development and can therefore only spend part of your time on providing paid support it clear that this is not making you rich!!


P