I picked a few of the questions you raised, the rest of my comments I have put in between.
1. Has the Framework gone so far "off course" with what people want the CMS to do ?
Good question, the fruit of the work done in the framework depends on the ability to use the framework code in Joomla. It can be argued that the framework can be used outside of the cms scope, but as we both know there are too many better alternatives already.
The only possible way to answer this question is try to refactor some of the framework code back into the CMS and see if it still fits while maintaining backwards compatibility. If it does the answer is : "no it hasn't gone to far off course".
2. Is namespacing something that should be considered in the CMS, and if not do we un-namespace code that has already been to fit back into the CMS architecture?
A debate over namespacing will draw away focus to move code from framework into CMS. Question is what comes first ? Or better said is namespacing a problem that needs to be solved before code from framework can be used inside the CMS. If the answer is no, then best approach is to un-namespace and use existing conventions using prefixes.
3. How do we continue to innovate in the active series without breaking compatibility in releases or stagnating development?
By adding incremental and fully compatible changes. Instead of using an approach to replace, use a approach of deprecation and legacy.
With 1.5 we moved all our legacy into the so called legacy layer which was nothing more then a plugin you could turn on or off. While technically having it enabled didn't have much effect - to our own surprise - people started to demand developers to run without "legacy mode". The market regulated itself from there.
FOF, RedCore, Nooku and others show that it's perfectly possible to keep 100% compatibility with Joomla core while providing heaps of innovation and progress. To do that, you don't need to replace existing core code, instead you add new code, and deprecate the old.
Inspiration can also be drawn here from EmberJS and their 2.0 strategy which focusses heavily on stability and progress without stagnation.
Rest of the comments are in between from here
On Sunday, October 26, 2014 4:30:41 PM UTC+1, Michael Babker wrote:
So I absolutely hate discarding the work people put into anything, especially when some of it is my own, so my question now is how can we salvage what has already happened?
Not sure if this is the right question to start with. I would prefer to start with a definition of a problem and then see what the best approach is. By narrowing down the problem, you can bring focus.
It doesn't necessarily need to be the first question, but to me it is a question that should be answered at some point. Has the Framework gone so far "off course" with what people want the CMS to do (and there are mixed opinions on that too, ranging from cries to stop adding features or trying to decouple extensions to looking at the architecture of the whole application stack) that none of the efforts the Platform/Framework produced are usable, or are there pieces that can be used? In terms of practicality to the CMS, some of the larger changes include refactored Event and Profiler packages and the Framework electing to deprecate the Log and Session packages, beyond that most of the remaining code is fairly close in structure and feature list to the comparable CMS classes. And there is the obvious difference in class organization between the two groups with one being namespaced; is namespacing something that should be considered in the CMS, and if not do we un-namespace code that has already been to fit back into the CMS architecture?
The Framework lost focus trying to be a general one size fits all tool, and in doing that much of our code was dumbed down to basic interfaces which aren't anything special. So can we refocus the work in the Framework and use it as a building tool for CMS 4.0? I say yes.
Don't see why not, but we both know CMS 4.0 is far off and the scope is very unclear. I rather focus on incremental change, if at some point you made so much change your rather call it 4.0 you can. This is how we went from 1.1 to 1.5. We did so much it made more sense to call it 1.5.
I'm in a spot right now where I want to focus on both short-term and long-term. In the short-term, the project is reaching a point where for the first time in four years it will have a single actively maintained release series and we have promised a support period as long as the lifetime of 1.5, so how do we continue to innovate in the active series without breaking compatibility in releases or stagnating development? The roadmap we published earlier this year certainly offers some ideas, but there's always more that can be done, especially as we begin to identify what the next major release will look like and how we can introduce a forward compatibility layer into the current series to assist in making adoption for the next series easier. As far as the long-term goes, I know that there is probably 2 or 3 years before we see a 4.0 release and I'd like to see ideas getting tossed around now and concepts start coming together for it sooner than later. My hope is that being able to have open discussions about where we as a community want to see the next release go would help to identify focus points and changes, both incremental and breaking, that could be incorporated into the appropriate development branch. What I'd like to not see happen again is that the branch for the next major release is established 6 weeks before the scheduled beta and a hurried rush to merge patches be made to throw a release together.
How can this be accomplished? Let's start with the MVC layer. The interfaces can be retooled (or dropped completely, who knows what the answer should be) to be supportive of the CRUD operations that we have in the CMS and we can slowly work to re-establish that code at the Framework layer.
Starting with the MVC layer is a good step. The MVC implementation and CRUD implementation forms the basis of most components, the easier and more convention based this works, the easier it will be for people to write their own components.
: In Nooku we have come a long way to improve upon this. Nooku implements a default CRUD flow that works completely out of the box. It's possible to create a working MVC triad with just a database table and a few lines of code to dispatch it. Resullt : a completely working JSON REST service. The http dispatcher
takes care of translation the HTTP methods to controller actions. JSON output is based on http://jsonapi.org/
standard as also used in EmberJS data layer.
I see a lot of Symfony in your dispatching and controller code. IMO that's an area they shine in. It actually works rather well once you have it implemented right, and from what I can see in the Koowa libraries it looks like all the generic stuff is covered well.
Code isn't based on Symfony though, they are
similarities but that's as far as you can go really. Symfony doesn't shine at all in the area of dispatcher and controllers. Symfony only exposes a crude base controller through it's framework bundle
The implementation is quite crude and looks more like an afterthought. No controller interface, no decoupling from the request etc which makes it nearly impossible to run Symfony as part of a proper HMVC architectture. In Nooku controllers and dispatchers are first class citizens of the architecture.
I'd like to seriously look at the CMS' rendering platform and how JDocument could be retooled, or even replaced with something like Symfony's PhpEngine Templating. Page output should be much easier to accomplish in multiple formats and having a strong View layer of our MVC coupled with a strong rendering platform will make it easier to provide our data in JSON, RSS, or HTML formats without some of the headaches that exist today. Along with that, I'd decouple aspects of JDocument and JHtml from the rest of the platform; the rendering engine takes full responsibility for all aspects rendering, be it reusable snippets or asset management, and the backend architecture focuses in primarily on data retrieval and management.
Correct, technically speaking it can be removed, the module and page head handling can be implemented using template filters which gives for a cleaner rendering pipeline. This also streamlines cache handling and solve some of the page cache issues.
This is an area Joomla really needs to step up and shine in. Not quite sure I've fully grasped this part yet but it looks promising.
Just an example to show you how we refactored away from using JDocument in a 100% compatible way while adding a lot more power under the hood. Our approach allows JDocument and our view pipeline to be used interchangeably.
I'd also suggest looking at how we can drop our singleton object storage with JFactory and move toward a Service Provider like environment. Having built some smaller scale Framework apps, I've found one instance where a singleton has been beneficial for me and ironically that singleton is my Factory class which is storing a reference to my DI container.
I wouldn't necessarily drop it, instead I would implement a DI below and make JFactory implement that. This gives you 100% backwards compatibility. You do indeed still need a singleton to get your DI. Chicken or the egg problem. No way around that.
In the context of Joomla, something like JFactory would always be a requirement. It isn't the best implementation, but the Factory object I'm using in my latest personal project is at https://gist.github.com/mbabker/0370d599ebfa5ed941f6
and that is instantiated with the application after my DI Container has been set up with my base services.
Factory a hardcoded DI, this is by design, as a dynamic DI created new challenges. The problem with a pure DI container is namespace collisions on the DI object identifiers (keys). A DI works if you have a relatively small amount of objects in your container, it fails if you have more, or if the object keys are defined at runtime. Technically you can consider J
Nooku solves this by using a standardised naming conventions for DI keys, we call them object identifiers. The DI itself is hidden quite deep inside the object manager and is not directly exposed. More info : http://guides.nooku.org/essentials/object-management.html
With that, moving forward on refactoring (where practical) places in the platform which are statics to create a more object oriented API. This was started with our unit testing sprint last week, and I think that move could help strengthen our APIs and improve the testability of them.
All in all, I feel like we can develop the Framework to be a strong toolset to build the next generation of the CMS on while still having the resources to be usable beyond Joomla.
I'm sceptic about this approach. "The best solution is found in the best definition of the problem
". 'Develop a framework to be a strong toolset for a next generation CMS to be usable beyond Joomla
' is not really a problem, more a vision. This is not different to what the platfrom and framework tried to do and this failed.
If there is anything that we can learn from 10 years of Joomla is that nothing 'next generation' has happened and is is likely to happen. Joomla isn't much different from Mambo, the core principles are still very much the same.
I would rather look at some of the architectural problems that Joomla has today and how you can solve them in a BC way to make live easier for extension developers. For users I would like to look at making things more simple again, and that means talking a about what to remove not what to add.
Joomla really needs to step up and address a lot of the architectural issues, and some of them may not be BC. For example, why does our MVC layer today have support for different controller classes based on the output format (like our controller.json.php files)? I'd rather this be dropped and controllers rewritten to work out of the box regardless of format.
Agreed. This was never part of the original architecture of 1.5. Format specific controllers where added later, unsure why. The implementation of is plain wrong. A controller decides based on the format what view to render. The format is part of the controller context (request), the action is the same being 'render' or 'display'.
Note : This is the same reason why a controller per action is a bad idea.
I'd like to see JError either killed off or reworked to be a useful error handling system; what it is today doesn't fit into that generalized description.
Agreed. Nooku solves this by standardising on using exceptions everywhere, we re-throw php errors
as exceptions. Error codes used are HTTP status codes, and if no code is given we fallback to 500.
Exceptions are dispatched as events using a special event exception publisher
. This allows any event subscriber to capture the exception and handle it, if no one handles it, it's being re-throw for PHP to handle it.
This makes exceptions very powerful mechanism. For example, we have a special event subscriber
that acts on Unauthorized
exceptions that are thrown. The subscriber will handle redirecting to the login screen and outputting the error message. Right now this is something that every component developer needs to handle himself. With Nooku this comes out of the box.
I'd also like for Joomla to not be a one-trick pony with its code (or code written for it) only being usable in a Joomla environment. It's true that users are still searching for full scale applications that will meet their demands with the simplest maintenance requirements possible, but developers aren't always implementing these applications anymore. Is Joomla's target market geared so much toward the end user looking for that easy to use application that we aren't trying to entice developers to use our platform, either by deploying the full CMS application or using parts of our API?
We have packages that may not fit into the current CMS structure (like our third party APIs) that could be beneficial to the greater PHP community.
Are those packahes solving problems the CMS has ? If not, just remove them.
Truthfully, much of that code never solved a CMS issue but predated a time when SDKs were prevalent on the marketplace. From what I can tell, our GitHub package is one of the few PHP packages wrapping their API with full coverage. Our Mediawiki package could be useful in the CMS if we were to build a stronger bridge to our docs wiki. Other packages though are at best developer resources.
Likewise, I can see us reaching a point where it could be practical to run applications in parallel to the CMS to create a more fine tuned environment while still utilizing CMS services. Imagine a web shop component running separate from the CMS, so it doesn't have some of the overhead of a standard CMS application cycle, but is still able to integrate with the CMS' user data, for example, to manage customer accounts. Enabling that could be a huge boost for Joomla in terms of creating more fine tuned and performant applications and extensions.
Is this really a problem Joomla should try to solve ?
Is it Joomla's responsibility? No. Is it something that interests me as someone who is willing to look beyond the CMS and see how Joomla code could be implemented in new ways? Yes. In many ways, the API is already there to support applications running in parallel (in theory the minimum you'd need is a separate index.php like file; application, menu, routing, and pathway classes; and register your application through JApplicationHelper::addClientInfo()) but it isn't a feature that most either realize is present or would be willing to write code for.
True, no need to pursue this.
Have I lost my mind yet or is this something that could actually be feasible?
Truthfully, I couldn't pick one. One of my downfalls has always been not being able to focus on one thing for an extended period because I'm always wanting to try something new or do more with what I already have. It took me two years to tag a stable version of my code wrapping the Transifex API for no other reason than I would continue refactoring internals or trying to do things better and not wanting to constrain myself by SemVer style rules.
Thanks for being honest. This is one of the hardest things you need to learn as a developer - especially if you wish to lead an open source project - There are new things happening every day and they all sound cool. The ability to make hard choices is key to the further success of Joomla. Without that, you get what you have now, a little bit of everything that really fits nothing.