Re: [jfw] Re: The Joomla! framwork for which applications?

So I absolutely hate discarding the work people put into anything, especially when some of it is my own, so my question now is how can we salvage what has already happened?

Not sure if this is the right question to start with. I would prefer to start with a definition of a problem and then see what the best approach is. By narrowing down the problem, you can bring focus.

It doesn't necessarily need to be the first question, but to me it is a question that should be answered at some point. Has the Framework gone so far "off course" with what people want the CMS to do (and there are mixed opinions on that too, ranging from cries to stop adding features or trying to decouple extensions to looking at the architecture of the whole application stack) that none of the efforts the Platform/Framework produced are usable, or are there pieces that can be used? In terms of practicality to the CMS, some of the larger changes include refactored Event and Profiler packages and the Framework electing to deprecate the Log and Session packages, beyond that most of the remaining code is fairly close in structure and feature list to the comparable CMS classes. And there is the obvious difference in class organization between the two groups with one being namespaced; is namespacing something that should be considered in the CMS, and if not do we un-namespace code that has already been to fit back into the CMS architecture?

The Framework lost focus trying to be a general one size fits all tool, and in doing that much of our code was dumbed down to basic interfaces which aren't anything special. So can we refocus the work in the Framework and use it as a building tool for CMS 4.0? I say yes.

Don't see why not, but we both know CMS 4.0 is far off and the scope is very unclear. I rather focus on incremental change, if at some point you made so much change your rather call it 4.0 you can. This is how we went from 1.1 to 1.5. We did so much it made more sense to call it 1.5.

I'm in a spot right now where I want to focus on both short-term and long-term. In the short-term, the project is reaching a point where for the first time in four years it will have a single actively maintained release series and we have promised a support period as long as the lifetime of 1.5, so how do we continue to innovate in the active series without breaking compatibility in releases or stagnating development? The roadmap we published earlier this year certainly offers some ideas, but there's always more that can be done, especially as we begin to identify what the next major release will look like and how we can introduce a forward compatibility layer into the current series to assist in making adoption for the next series easier. As far as the long-term goes, I know that there is probably 2 or 3 years before we see a 4.0 release and I'd like to see ideas getting tossed around now and concepts start coming together for it sooner than later. My hope is that being able to have open discussions about where we as a community want to see the next release go would help to identify focus points and changes, both incremental and breaking, that could be incorporated into the appropriate development branch. What I'd like to not see happen again is that the branch for the next major release is established 6 weeks before the scheduled beta and a hurried rush to merge patches be made to throw a release together.

How can this be accomplished? Let's start with the MVC layer. The interfaces can be retooled (or dropped completely, who knows what the answer should be) to be supportive of the CRUD operations that we have in the CMS and we can slowly work to re-establish that code at the Framework layer.

Starting with the MVC layer is a good step. The MVC implementation and CRUD implementation forms the basis of most components, the easier and more convention based this works, the easier it will be for people to write their own components.

Nooku : In Nooku we have come a long way to improve upon this. Nooku implements a default CRUD flow that works completely out of the box. It's possible to create a working MVC triad with just a database table and a few lines of code to dispatch it. Resullt : a completely working JSON REST service. The http dispatcher takes care of translation the HTTP methods to controller actions. JSON output is based on http://jsonapi.org/ standard as also used in EmberJS data layer.

I see a lot of Symfony in your dispatching and controller code. IMO that's an area they shine in. It actually works rather well once you have it implemented right, and from what I can see in the Koowa libraries it looks like all the generic stuff is covered well.

I'd like to seriously look at the CMS' rendering platform and how JDocument could be retooled, or even replaced with something like Symfony's PhpEngine Templating. Page output should be much easier to accomplish in multiple formats and having a strong View layer of our MVC coupled with a strong rendering platform will make it easier to provide our data in JSON, RSS, or HTML formats without some of the headaches that exist today. Along with that, I'd decouple aspects of JDocument and JHtml from the rest of the platform; the rendering engine takes full responsibility for all aspects rendering, be it reusable snippets or asset management, and the backend architecture focuses in primarily on data retrieval and management.

Correct, technically speaking it can be removed, the module and page head handling can be implemented using template filters which gives for a cleaner rendering pipeline. This also streamlines cache handling and solve some of the page cache issues.

Nooku : In Nooku I have implemented a view layer. The view layer has support for following format : html, json, rss, vcard, csv. The html format makes use of the template layer to render templates. See : https://github.com/nooku/nooku-framework/tree/master/code/libraries/koowa/libraries/view The JSON format works out of the box, if you connect it to a model is can render the model data as JSON. More info : http://guides.nooku.org/json.html

The template layer implements a complete template engine with support for : tags, functions, helpers and filters. It also offers an abstraction layer for other template engines with out of the box support for Twig, Mustache and Markdown. See : https://github.com/nooku/nooku-framework/tree/master/code/libraries/koowa/libraries/template One of the cool features is that it's possible to use different formats interchangeably in templates.

Nooku implements it's own 'page' template which is rendered without using JDocument. We use this to render pages that do not need to be rendered inside the Joomla template, instead of tmpl=component. The page template shows how we use template filters to inject assets and head information into it. https://github.com/nooku/nooku-framework/blob/master/code/libraries/koowa/components/com_koowa/views/page/tmpl/koowa.html.php

This is an area Joomla really needs to step up and shine in. Not quite sure I've fully grasped this part yet but it looks promising.

I'd also suggest looking at how we can drop our singleton object storage with JFactory and move toward a Service Provider like environment. Having built some smaller scale Framework apps, I've found one instance where a singleton has been beneficial for me and ironically that singleton is my Factory class which is storing a reference to my DI container.

I wouldn't necessarily drop it, instead I would implement a DI below and make JFactory implement that. This gives you 100% backwards compatibility. You do indeed still need a singleton to get your DI. Chicken or the egg problem. No way around that.

Nooku : In Nooku I have build an object manager which implements both decency injection and a service location pattern. By combining both I can use the benefits of both without the drawbacks. Since each object in Nooku extends from the same base object, the object manager can instantiate any object from anywhere. See : http://guides.nooku.org/essentials/object-management.html

In the context of Joomla, something like JFactory would always be a requirement. It isn't the best implementation, but the Factory object I'm using in my latest personal project is at https://gist.github.com/mbabker/0370d599ebfa5ed941f6 and that is instantiated with the application after my DI Container has been set up with my base services.

With that, moving forward on refactoring (where practical) places in the platform which are statics to create a more object oriented API. This was started with our unit testing sprint last week, and I think that move could help strengthen our APIs and improve the testability of them.

All in all, I feel like we can develop the Framework to be a strong toolset to build the next generation of the CMS on while still having the resources to be usable beyond Joomla.

I'm sceptic about this approach. "The best solution is found in the best definition of the problem". 'Develop a framework to be a strong toolset for a next generation CMS to be usable beyond Joomla' is not really a problem, more a vision. This is not different to what the platfrom and framework tried to do and this failed.

If there is anything that we can learn from 10 years of Joomla is that nothing 'next generation' has happened and is is likely to happen. Joomla isn't much different from Mambo, the core principles are still very much the same.

I would rather look at some of the architectural problems that Joomla has today and how you can solve them in a BC way to make live easier for extension developers. For users I would like to look at making things more simple again, and that means talking a about what to remove not what to add.

Joomla really needs to step up and address a lot of the architectural issues, and some of them may not be BC. For example, why does our MVC layer today have support for different controller classes based on the output format (like our controller.json.php files)? I'd rather this be dropped and controllers rewritten to work out of the box regardless of format. I'd like to see JError either killed off or reworked to be a useful error handling system; what it is today doesn't fit into that generalized description.

I'd also like for Joomla to not be a one-trick pony with its code (or code written for it) only being usable in a Joomla environment. It's true that users are still searching for full scale applications that will meet their demands with the simplest maintenance requirements possible, but developers aren't always implementing these applications anymore. Is Joomla's target market geared so much toward the end user looking for that easy to use application that we aren't trying to entice developers to use our platform, either by deploying the full CMS application or using parts of our API?

We have packages that may not fit into the current CMS structure (like our third party APIs) that could be beneficial to the greater PHP community.

Are those packahes solving problems the CMS has ? If not, just remove them.

Truthfully, much of that code never solved a CMS issue but predated a time when SDKs were prevalent on the marketplace. From what I can tell, our GitHub package is one of the few PHP packages wrapping their API with full coverage. Our Mediawiki package could be useful in the CMS if we were to build a stronger bridge to our docs wiki. Other packages though are at best developer resources.

Likewise, I can see us reaching a point where it could be practical to run applications in parallel to the CMS to create a more fine tuned environment while still utilizing CMS services. Imagine a web shop component running separate from the CMS, so it doesn't have some of the overhead of a standard CMS application cycle, but is still able to integrate with the CMS' user data, for example, to manage customer accounts. Enabling that could be a huge boost for Joomla in terms of creating more fine tuned and performant applications and extensions.

Is this really a problem Joomla should try to solve ?

Is it Joomla's responsibility? No. Is it something that interests me as someone who is willing to look beyond the CMS and see how Joomla code could be implemented in new ways? Yes. In many ways, the API is already there to support applications running in parallel (in theory the minimum you'd need is a separate index.php like file; application, menu, routing, and pathway classes; and register your application through JApplicationHelper::addClientInfo()) but it isn't a feature that most either realize is present or would be willing to write code for.

Have I lost my mind yet or is this something that could actually be feasible?

Anything can be done. "Deciding what not to do is as important as deciding what to do" - Steve Job. If today you had to pick one of the above items - to start working on tomorrow - which would it be ? Why ?

Truthfully, I couldn't pick one. One of my downfalls has always been not being able to focus on one thing for an extended period because I'm always wanting to try something new or do more with what I already have. It took me two years to tag a stable version of my code wrapping the Transifex API for no other reason than I would continue refactoring internals or trying to do things better and not wanting to constrain myself by SemVer style rules.