Tobias Macey: yeah so thanks to everyone for taking the time to join today and for putting together the DEMO videos.
Tobias Macey: don't have a sort of strict agenda, because this is the first meeting so it's mainly just kind of exploratory to see what do we want the sort of goals and mandate of this working group to be.
Tobias Macey: So it's figuring we can start off by just having a sort of general discussion about what are the different.
Tobias Macey: systems and methods that people are using for analyzing that data and maybe focusing on.
Tobias Macey: What is the overall focus of that given analytic solution, and who are the target users and then we can use that as a launching point for discussing.
Tobias Macey: You know what are the complexities or challenges that we're faced and building these different solutions, what are the.
Tobias Macey: sort of common elements that exist across all of them, and how can we sort of align ourselves to making it easier for people to be able to build analytics solutions for their different edx deployments.
Tobias Macey: So that's sort of what I had in mind, so I guess i'll let folks start.
Tobias Macey: Discussing sort of the first couple of questions there of the systems that you've built or used to having to do with that X data, what are the sort of general goals of the system and who are the target users.
Régis Behmo: Right procedural question toby do is rather.
Edward Zarecor: Would it be helpful for for me to start talking to take some notes.
Tobias Macey: yeah certainly doesn't hurt i've got sort of rough sketch notes here that i'll probably be filling out to, and then we can maybe merge them and put them in the wiki when we're done.
Tobias Macey: reggie so what's it look like you're about to start saying something.
Régis Behmo: yeah some I came here because I work on care that's no secret.
Régis Behmo: Advertising it's maybe a bit too loudly.
Régis Behmo: Anyway, so, so what i'm what i'm trying to, I would like to get from this working group is ideas on what kind of analytics you guys are interested, I think it's to be great if we could come to.
Régis Behmo: Like.
Régis Behmo: Standard reports that you think are valuable for.
Régis Behmo: Educational teams and I would like to be able to translate those visions into actual Karen implementations so actual dashboards from inside Karen.
Régis Behmo: I have my own perception of what I want analytics to be and it's like a dedicated.
Régis Behmo: platform, separate from open X, which draws its data from from the next.
Régis Behmo: But maybe there are different visions, maybe it would make sense to integrate analytics better insight open X.
Régis Behmo: So that's just just just an idea that that's that's where i'm at anyway.
Tobias Macey: And as you were to architected the Karen system i'm wondering if you had a particular sort of user persona in mind as to who would be best served by it and sort of what types of skill sets they might be, they might have for being able to take advantage of it.
Régis Behmo: yeah so.
Régis Behmo: The the gold one of the goals that I had in mind when I.
Régis Behmo: first started researching what would be the building blocks for for Karen and I really wanted to to provide analytics and analytics solution that would be useful, both for our users, so people know what they want to do with that data and stats and also regular core staff might not.
Régis Behmo: have great analytic skills, but still might want to to to to get data, but their course so visualizations that would really be.
Régis Behmo: Given many insights about how to run their course.
Régis Behmo: So those two were the main.
Régis Behmo: Targets I also had a more business oriented.
Régis Behmo: target in mind, which was to sell provide care to open an xm providers.
Régis Behmo: Then themselves build visualizations and dashboards for their own users, because they know their users very well and I don't so these people should be able to design new dashboards for their end users.
Tobias Macey: Okay.
Tobias Macey: i've got lots of other things we can dig into, but I think we'll save that for the sort of general debate about sort of common elements.
Tobias Macey: So I guess who wants to go next.
Andrés González: Okay, so like I can talk about our approach.
Andrés González: That we call panorama the main difference, I think, in the in the architecture is that it is not development, it is not an application application.
Andrés González: or anything like that it is based on aws services so basically what we do is extract the data for data us very simple csv files that can then be analyzed by.
Andrés González: By the service.
Andrés González: and
Andrés González: Maybe that the key target is.
Andrés González: Any any kind of user, so we can develop.
Andrés González: That was for in user students for teachers for.
Andrés González: academia administration stuff or even service providers.
Andrés González: Because of this approach, we can even nourish several open assistances into one single dashboard so we can administer many installations in in the same place, and also another.
Andrés González: Another key point is that, if it can integrate with third party data sources so if you have another database, or whatever you have.
Andrés González: We can import the data into the structure and then, if you link course ID or student ID you can.
Andrés González: get a.
Andrés González: richer dashboard with information from other sources.
Andrés González: and
Andrés González: I think that's the main well.
Andrés González: Another another.
Andrés González: important point is that the end users are able to create their own dashboards because we expose the data.
Andrés González: Either raw data or semi processed data, and then the end users can construct whatever dashboard they want, so I think flexibility in in report creation is another key point for an analytic system.
Tobias Macey: yeah.
Esteban Etcheverry: So just just want to add another thing about our approach right good undress with with been around and so.
Esteban Etcheverry: In terms of the target users right, so we when we started thinking about the tool that we get and it will be getting lots of requests from.
Esteban Etcheverry: Our customers that they want to have like an overall view of what happens in the edx platform, and this means not only sometimes a view.
Esteban Etcheverry: In terms of the process, but sometimes, especially the view in terms of each students by what you do you take one student and you want to see the overall activity in this new courses and even ratings that.
Esteban Etcheverry: So we that there, there are different types of users of an analytic to right sometimes a stretch saying right administrator as one one they want to do some things right, but maybe we should think.
Esteban Etcheverry: about right.
Esteban Etcheverry: Define I will find like like power users right out or even instructor try, because sometimes the struggles, they they they do have information all the posts that if they want to focus on the students right and the overall activity.
Esteban Etcheverry: that's like a different approach, and I think that that's one of the points that will really, really provide value to the to the to the opening screen experience in terms not none of the power users, because maybe if there's a.
Esteban Etcheverry: As register saying like amanda analytic tool, the runs in parallel run because of different dynamics and takes data from from the blanks platform, but if you if you.
Esteban Etcheverry: see the point of view of the of the structure or maybe.
Esteban Etcheverry: director right, I want to see what happens with the students in the platform, it will be very valuable if they can easily have access right maybe embedded into the platform right or some somewhere like that.
Esteban Etcheverry: The history of a student right because the boys a direct impact on on stuff.
Andrés González: And another thing and another thing I didn't mention is it blood ability to another machine learning, for example, a capabilities, so you can build another another system to to enrich the data with machine learning artificial intelligence and an increase the this kind of functionality.
Miranda von Salis: Because we go a little bit to what Esteban was saying we up I work for as may learning solutions we.
Miranda von Salis: build our own courses, on behalf of universities and so most our main stakeholder is the success team within our company, who is you know.
Miranda von Salis: assigned to certain courses and they really want to know what learners are doing in the course how far they've done, and then the curriculum team wants to know.
Miranda von Salis: What content are they interacting with the best and being able to compare that across courses ones that had certain you know.
Miranda von Salis: As five P, if things put in, or some of those other like widgets that we're testing out so it's all about trying to like see the impact of learning of specific elements of the course as well as tracking learners over time.
Miranda von Salis: And then, of course, having you know clean data that we can compare even if major course changes are made.
Miranda von Salis: Across time in our database we're in snowflake.
Miranda von Salis: and using that to connect to third party data we track sort of we have our own version of zoom sort of them, we can see what teams are when teams are meeting how those meetings are going, and we want to be able to merge that in and as well as some other things so.
Miranda von Salis: And then, again, also to be able to do some kind of machine learning people are very interested in personas who is taking our course who is succeeding or.
Miranda von Salis: most satisfied and that kind of feeds back into our own sort of company image, what kind of.
Miranda von Salis: courses are we running and so it's being able to highlight certain elements learning objectives themes that people think are being built into the course and being able to tie that back in so there's a lot of sort of data merging that needs to happen and.
Miranda von Salis: You know it's we have university partners who need to see their very kind of curated view faculty staff faculty curriculum marketing student success, you are all trying to get their own sort of view.
Miranda von Salis: And that's where like.
Miranda von Salis: You know, having that flexible data and.
Miranda von Salis: You know there's some things that we just you know you're asked for and you're like i'm not even sure how we can get that out, you know.
Miranda von Salis: So.
Régis Behmo: Somewhere that can kick, can I ask what what solution you're using right now, if any.
um.
Miranda von Salis: well.
Miranda von Salis: i'm not entirely sure how to answer that question so.
Miranda von Salis: We, how are we Okay, of course, they're built through open craft logs and greed reports are dumped into s3 those get piped I pipe them into snowflake parse them out to useful tables or analysts can build stuff out into tablo or other things.
Miranda von Salis: You know i'm I need to start using DVD more I like i'm a little babies are that.
Miranda von Salis: Just because we're scaling so fast that some of this is just not we know and the iteration is the course designs don't match up the same way, so a lot more transfer transformation is needed going forward.
Miranda von Salis: But we're we're pretty in house.
Miranda von Salis: I don't know if I answered your.
Miranda von Salis: I mean that's, that is, the thing right though.
Miranda von Salis: Is that there are startup you know, even if we're in burgeoning startup phase now who we've been around about two years, who aren't necessarily contracting with an analytics team.
Miranda von Salis: You know that knows.
Miranda von Salis: All the ins and outs and being able to have whatever we produce be welcoming to folks who are trying to not screw it up until we get paid enough to to have you know calling the big guns is.
Miranda von Salis: I think pretty important in terms of like open open edx fs you know.
Tobias Macey: So I guess now if folks have sort of common challenges or barriers that they ran into well building either an analytics.
Tobias Macey: solution or doing hunger and analytics and.
Tobias Macey: Some of their elements.
Esteban Etcheverry: Yes, I have.
Esteban Etcheverry: Another comments.
Esteban Etcheverry: Missouri injuries if it's not i'm not the technical guy right so sorry fun.
Esteban Etcheverry: out, we should work on a goal right of this 14 in general terms, is because we were talking about or some of you mentioned our natural right.
Esteban Etcheverry: And I think I mean this is never gonna be an idea that i'd say 510 typical requests right that they can be transformed to one to 510 to do dashboards but it's in the tool needs to have the capability to build custom views or custom dashboard right because.
Esteban Etcheverry: We have me he.
Esteban Etcheverry: has different goals right some some of them, they sell courses and they want to see.
Esteban Etcheverry: The e commerce results, and they want to link it with the students and they want to see.
Esteban Etcheverry: who's assume that hubspot more and more courses, or whatever, and some others that don't serve and they're going to see participation right, so this is not this never going to be just one answer for every customer, so we.
Esteban Etcheverry: should work on something that has maybe like like a set of initial templates it's we can maybe discuss on agree, we might find like the I don't know 10 100 or more typical requests right.
Esteban Etcheverry: And then we should work on some something that it's like as this flexibility to build your own dashboard.
Miranda von Salis: And the underlying data sources right like.
Miranda von Salis: letting people fend on their own, and then they plop it into the dashboard and it won't work right and.
Miranda von Salis: And I think that's that can be.
Miranda von Salis: Part of that.
Tobias Macey: Okay.
Tobias Macey: See I think makes sense now to talk about some of the sort of challenges that are posed by sort of the format of the data sources that we're working with.
Tobias Macey: Being able to sort of collect and transform them some of the contextual knowledge that's necessary to be able to effectively build dashboards and analytics based on that data so curious what folks experience or their.
HIV.
supervisor sorry.
Tobias Macey: that's okay.
Edward Zarecor: there's there are some like more Meta points I wanted to like put on the radar.
Tobias Macey: folks are kind of Dan working group.
Edward Zarecor: I think what you're talking about in terms of like issues with the data is really closely related to one of the Meta points like when I think about what this group can do.
Tobias Macey: is providing feedback loops into the development and in continued development of the open edx platform to ensure that we're improving the way that we provide you know data API is for people who are trying to do in analytics so.
Edward Zarecor: Like from a medical point of view that's one of the things I would love this group to be able to do so, I think we'll get to that in just like one second.
Edward Zarecor: There are some other places where you know I.
You know I want this group to evolve into like making other sorts of architectural and kind of.
cool recommendations like one of the things that's like right in front of us now is like what we're going to do about elasticsearch open search.
So i'd love this to be a group, where we can have a robust conversation about what the open edx project should do around open search elastic search and come in, maybe make a recommendation out to the broader project about what the next step up.
Edward Zarecor: And then another thing that I had on my mind was in a room, he said mentioned that he thinks like having a separate system from open at X for data makes sense.
and
Edward Zarecor: To a degree, I agree with him, but I also think that the long term vision of data and open edx should be that it is readily available for the platform to make decisions for learners and educators, in terms of recommending content or deploying an activity.
Edward Zarecor: And also that analytics would be in context that they wouldn't necessarily be.
Edward Zarecor: A separate site that you go to like insights is today, but you know, an instructor might be able to click on a link next to an xbox and see like the details about how to use rarity and.
Edward Zarecor: So i'm not arguing for like a tight coupling between the data systems and the platform, but.
Edward Zarecor: Your paths for integrating that data back into the kind of experience of learners so it's like a couple of you know so feedback loops helping the project make decisions about data related to things and a future vision of.
Tobias Macey: How the platform should evolve to better use data everywhere.
Tobias Macey: yeah and I think that your third point, there is definitely exceedingly valid, especially given the recent rise of the sort of operational analytics category with tools like sensors or high touch where you're pushing data.
Tobias Macey: From the data warehouse back into the systems where the data was sourced from, and I think that a similar.
Tobias Macey: architectural elements would be viable for the edx platform where we're pulling the data from edx into an analytical system, whether that's something that's actually baked into the edx platform for sort of lightweight analytics or a more sort of full fledged.
Tobias Macey: analytical environment where we can build out sort of these common transforms and common.
Tobias Macey: metrics that can then be fed back into the you know edx platform either via you know, a push mechanism from the analytical system or by exposing api's or sort of common interfaces for edx platform to be able to pull from that analytical system.
Tobias Macey: yeah no totally not not currently opinionated about what the design, should be, and you know, probably api's makes more sense than like a data, you know data Mart in the platform.
But I think it was Miranda.
mentioned the GDP and.
standardized I think.
Tobias Macey: My intuition, is that.
Edward Zarecor: It would be valuable for us to you know to standardize on kind of the the format of the models.
Tobias Macey: Right, you know.
It would be great if, for example, if Karen and panorama have the same idea of what an a role in enrollment was right and that turns out to be more complicated than it.
needs to be.
Tobias Macey: But you know I think like that's a place where like if we agree on kind of what the fundamental models are that we're using and the analysis that would be yeah.
Tobias Macey: yeah and that keys on to another sort of recent trend that's been popping up in a number of places is the idea of the.
Tobias Macey: sort of semantic layer as a distinct element of the architecture, where you have the.
Tobias Macey: metrics as a sort of common definition that you can then consume, either in the business intelligence system or by api's so there's a company transform that came up that started up recently.
Tobias Macey: there's an open source project called metrical that sits that actually hooks into DDT and is able to expose metrics as a sort of common element that you can either.
Tobias Macey: You know, you can query it a sequel or are there other interfaces to be able to access those, and so I think that something in that they.
Tobias Macey: From an architectural perspective would make sense for being able to have that common definition of this is what an enrollment means and then be able to query that from edx platform as an API.
Tobias Macey: So yeah I guess given Those sort of Meta elements i'm happy to shuffle around the agenda folks want to dig into any or all of those or if we should dig more into sort of.
Tobias Macey: The sort of common requirements for data projects or challenges that folks are facing in sort of collecting and processing the data.
Régis Behmo: yeah so so one of the challenges that I faced.
Régis Behmo: very early in the process is that obviously in opening next we need to draw data from.
Régis Behmo: different sources, which are the origin is.
Régis Behmo: So.
Régis Behmo: Different that there are differences, not only in how you access data but also how you how the data is structured so.
Régis Behmo: In the case of Karen that there are tracking logs.
Régis Behmo: My sequel data, and they are stored in mongodb it's i'm thinking about the course structure in mongodb and the first.
Régis Behmo: The first two I managed to to solve the problem very early of of how they were they were separated it and I, I think I managed to to merge them in a rather practical database.
Régis Behmo: But mongodb the structure that nature of the mongodb databases and open it makes it more difficult to to access data there, so when, for instance, we need to access the name of course units, you will need to have a rather console consult a mongodb query to fetch it.
Régis Behmo: So.
Régis Behmo: I did not find a very elegant solution to that problem I I just.
Régis Behmo: Do a batch processing of mongodb where I just import all of the.
Régis Behmo: Sub sub section names for mongodb to to click house, which is the final a database that source all the analytics data and so it's not very elegant.
Régis Behmo: If.
Régis Behmo: If all were if everything was was stored in a more structured manner, it would be easier to access the data.
Régis Behmo: Good.
Andrés González: We have some points in common, I think we agree that the main sources of information or the relational that mysql database, the mongodb and the tracking logs I think we agree that a mysql is the easiest part because data it's easy to extract is structure.
Andrés González: And we also face some issues with mongodb we our approach was a creating a managed comments, so we are extracting the data from Python and not directly from mongo db.
Andrés González: Little was kinda had that it works, the.
Régis Behmo: same here same here.
Andrés González: yeah and the the the the real challenge that we face was processing the tracking logs because.
Andrés González: We found several problems, and I think this is something maybe the edx team can help.
Andrés González: Some of the trucking log.
Andrés González: files has some corrupt data, maybe because the lines were too long, it wasn't really a json world formatted structure so.
Andrés González: We have to fix some errors there with data and then.
Andrés González: And I think there is no standard around the tracking logs.
Andrés González: You know, several pieces of software send logs in its own way and.
Andrés González: Each each process.
Andrés González: managed in in in a different way, so it is very difficult to to get something structure from although struggling locks I think there is something that can be.
Andrés González: done, they are in in how logs are registered, and it is a very interesting source of information, because I am it records everything that students do.
Andrés González: As Marian dimension that they want to know the activity, what the students are consuming what kind of X blog or what kind of content are our people consuming and when and how long so all this information is in the tracking logs so.
Andrés González: I think if we could find a way to standardize the way that the events in the blood for a lot in in this files would help a lot and then extracting and processing the data.
Miranda von Salis: When it comes to tracking logs the fact that we have these block ids, but I have yet to find a way to export any type of look like block ID to location.
Miranda von Salis: So I don't even know what module any of these block ids are in and I can't even use it for that we use Google analytics to figure out what my module content page duration that kind of stuff because you can't even like.
Miranda von Salis: You know, in theory, you could put in sequence or users all their cliques and activity and figure out how long they were in the system and on each page except you can't map back to that so.
Miranda von Salis: Any kind of way of having that even as a an occasional export it doesn't even have to be in the log necessarily but to crosswalk it back would be.
Miranda von Salis: ideal, and then I run into all the time, you know it tracks all the events and it tracks, when someone does a search when the results are displayed and when they select a search result.
Miranda von Salis: This is my current my today this week's issue, but not what the search term was I can't figure out where the where the keywords are going right but that's important what what our learners struggling with and if we're adding to those that are non searchable because they're all in Java.
Miranda von Salis: Or we depriving are we depriving a certain subset of users who really rely on that tool and so like something like something where there's there's additional detail.
Miranda von Salis: In any of these trackers or even again as where is that going, I mean you know these queries and how can, how can they be accessed is like.
Miranda von Salis: And then we can match it up back you know, based on the the you know event ID or or whatever right like who did it and when, but like darn you know.
Tobias Macey: yeah.
Tobias Macey: I think tracking logs has definitely been one of the thorniest problems for a long time, and having a standardized schema that they need to be able to map to would go a long way and solving a lot of problems in that regard.
Andrés González: yeah.
Andrés González: Absolutely.
Tobias Macey: And then.
Tobias Macey: Another element of that is that there's been the recent introduction of.
Tobias Macey: sort of events and hooks as a new system where there is a registered schema in a separate repository that gets installed into edx platform and i'm unclear as to sort of.
Tobias Macey: What level of overlap or collision might happen or exist between the existing tracking logs in this new inventing system and sort of what the long term looks like in that regard.
Tobias Macey: You see, you smiling and I don't know if you have any insights.
Edward Zarecor: No, sorry I was smiling because when I typed your initials and converted it to trademark and.
Edward Zarecor: totally unrelated.
Tobias Macey: All right, yeah.
Régis Behmo: I think the the new hook event system that edge next contributed and actually plugs a fairly elegantly with existing.
Régis Behmo: tracking log system because it's just a way of.
Régis Behmo: Adding callbacks to the code, so it should be able to to be possible to.
Régis Behmo: To to add callbacks that ad tracking lads should be possible to modify the tracking system to make use of that new callback system so, although the two are not connected, yet I think they can play great together.
Edward Zarecor: Okay yeah I would agree with that and I think there's a third kind of architectural gesture going on now, which is all things back together, which is the message bus work.
You know, maybe eventually the hooks and signals end up on a message bus maybe eventually the event stream ends up in the message bus.
And you know it gives us a great deal of kind of more flexibility, but to have these things all separate and not have a plan, where were the factors come.
Together, would be would be bad, but I think getting back to kind of the Meta recommendations, I think that this is probably a great group of people to start thinking about thinking about how we want those things to come back together and kind of sport, you know.
For each other.
Tobias Macey: yeah and one other element of that sort of Meta question is, you know you know what one of the sort of long running.
Tobias Macey: Aspects of analytics for the open edx community is insights which has been.
Tobias Macey: sort of the fate has been unclear for a while now as to sort of who owns it what the you know whether it will continue to be supported, or evolved, whether we should just.
Tobias Macey: drop support for it entirely and if there can or should be a Community supported sort of default installation for analytics.
Tobias Macey: So I guess that's another thing that's worth at least keeping in the back of our minds as we continue with these meetings so I don't have an answer to any of those but.
Tobias Macey: I figured i'd call out the elephant in the room.
Régis Behmo: yeah so.
You.
Régis Behmo: Know it's.
Régis Behmo: Obviously, when I started working on character, I thought long and hard about Okay, should I it's going to be open, should it be open source should be free, or should we should pay a commercial product and it plugs nicely into Twitter so.
Régis Behmo: It would have made sense to just give it away for free.
Régis Behmo: But I figured that's another takes is a feature that is.
Régis Behmo: Most important for commercial enterprises and it makes sense to ask them to to to buy product for something that is core to their to their own business.
Régis Behmo: And, and I also wanted to address the mutations about insights not just about the maintenance, the project, but the fact that.
Régis Behmo: Inside has was built a couple of years ago when.
Régis Behmo: Not all the pieces of the clothes, that we have today were available at the time, at the time hadoop was.
Régis Behmo: cutting edge.
Régis Behmo: Maybe now there.
Régis Behmo: There are better ways to to achieve things.
Régis Behmo: I don't I don't have a very good conclusion to this this paragraph.
Régis Behmo: That that's what I have to say.
Tobias Macey: I don't think you need to hedge with the maybe I think he can say definitively that there are better ways to do things, then who do these days.
Edward Zarecor: I mean i'll make a provocative statement, and you know, being in the from the coming from the other side, maybe, particularly provocative but.
Edward Zarecor: I don't think insights as a future to be perfectly honest with you.
And I think we need to acknowledge that, so I think that.
He could have a future in some respects, like, for example, if we got rid of the Duke and replace the model building with like db T or something like that and constructed.
analytics API where like data mark is very simple it's basically just key value pairs we could build that using you know, DVD relatively easily.
Edward Zarecor: But it's not it doesn't give people the ability to do self service creation of visualization doesn't give you the ability to visualization.
Régis Behmo: directly in DR mess or studio does a lot of gaps.
I think at X or to you is unlikely to death in it.
Because I think everybody.
Edward Zarecor: This was just called the elephant by name, we realized that week frogging the decisions that were made insights is a better decision and trying to get up to enterprise.
Edward Zarecor: Anybody disagrees with that, let me know, but you know.
Edward Zarecor: On the tech side, I feel like it's it's that, even if it is a little bit.
Tobias Macey: yeah I think that trying to iterate from insights to what a sort of ideal sort of Compostable solution would be.
Tobias Macey: Is a much longer and more time and Labor intensive investment that anybody cares to imagine, and I agree that leapfrogging is the more realistic decision.
Tobias Macey: And, given the fact that insights has been largely unsupported for some time now, I think that.
Tobias Macey: We can even reasonably say that there isn't going to be sort of an upgrade path for people who are using insights other than you know customer tl and i'm sure that there's a decent consulting opportunity for a number of people in the Community or.
Tobias Macey: for helping people with that credential migration.
Régis Behmo: So I think it's perfectly fine if X job support for insights i'm not just saying that because I actually selling a competitor to between sites, but it's it's really important that Alex.
Régis Behmo: says this publicly and and not have people wait for.
Régis Behmo: insights for for an upgrade to insights forever.
Régis Behmo: And all and not come to open an ex with false expectations, and I think it's good if.
Régis Behmo: X follows follows through with the application process it's perfectly fine if you guys admit hey we don't know how to maintain this anymore, just like it happened recently with the E commerce.
Régis Behmo: And I think it's Okay, if you drop support for something just need to keep to be transparent, with the Community about it.
Edward Zarecor: or i'll double check, but I think there's been like that for tickets open for insights for like a year, at least.
Edward Zarecor: And also, I mean, I think that.
Edward Zarecor: One of the reasons why I kind of a Community group or you know the leaders in the analytic space get together and talk about what the neck with the plan for the project is is that you know X.
Is it like, is it the only stakeholder here right so.
Edward Zarecor: What I want us for us to have a.
conversation.
What open edx should offer as in terms of analytics if anything right.
Tobias Macey: Like one option would be everybody wants to have their own commercial solution and open at X, just as api's pollution and other one is that it's a little bit more batteries included in the default.
analytics experience that's available that actually works and is, you know skills ups.
Edward Zarecor: I have to be honest with you like, when I think about it from a project point of view that makes more sense to me than having nothing.
I just think like a educational platform with no calls analytics experience, particularly attractive to the market.
For us, that kind of crashes into people's.
virtualization plan so like knowledge that too.
yeah.
Tobias Macey: yeah and I think you know as far as like an out of the box.
Esteban Etcheverry: capability.
Tobias Macey: So sorry go ahead Esteban.
Esteban Etcheverry: My the only one who's having like.
Esteban Etcheverry: What I was saying.
Esteban Etcheverry: The.
Esteban Etcheverry: I think it's.
Esteban Etcheverry: it's a relevant.
Esteban Etcheverry: conversation here and.
Esteban Etcheverry: better now.
Tobias Macey: Yes.
Esteban Etcheverry: Okay sorry i'm having issues with my Internet connection, so the thing is that.
Esteban Etcheverry: I think.
Esteban Etcheverry: some kind of analytic again sorry i'm technically.
Esteban Etcheverry: might not be talking properly, but.
Esteban Etcheverry: As the overall looking for the growth of the open index project as a whole.
Esteban Etcheverry: It should have some kind of.
Esteban Etcheverry: At least with with with a basic like.
Esteban Etcheverry: analytic.
Esteban Etcheverry: Information dashboard uses within the the open the Open Source option right like like understand our.
Esteban Etcheverry: And organization right, I think there will always be opportunities, I mean there will always be opportunities for for customization adaptation rights or.
Esteban Etcheverry: Different businesses different organization needs, I think, even if we have.
Esteban Etcheverry: Like a set of let's say 10 dashboards that are out of the box from from the new solution that will always be able to tease for some commercial endeavours to customize.
Esteban Etcheverry: Things because there are different organizations, I mean we tend to get requests to connect with salesforce sometimes we stop I mean there's no way the native installation of the native to fulfill the least in in governance platform will cover all of that, so I think.
Esteban Etcheverry: What is this is, I think this is a bottle of this discussion, and I think again, because here I mean we're focuses on the analytics to this group.
Esteban Etcheverry: But, again, I mean, sometimes in the marketing group, we have different discussions like So what do we stand us open others in the elements world right.
Esteban Etcheverry: So they're using the whole picture and there are other tools right some of those are are not open source right but, again, the customers they usually don't care, then the solutions right, so I think if we could incorporate some some way of like.
Esteban Etcheverry: native and engaging information already using the platform right.
Esteban Etcheverry: That will be better for the.
Esteban Etcheverry: position of an expert in the finance world.
Tobias Macey: yeah.
Tobias Macey: And as far as sort of the.
Tobias Macey: Out of the box situation you know, there are a number of folks who aren't represented here, currently, you know, one of the things that has been around for a little while but has been.
Tobias Macey: had some troubles and staying up to date with the releases is the figures project which is sort of you know, minimal out of the box solution of you know, here are some lightweight analytics based on the data that lives in my sequel.
Tobias Macey: not saying that we should necessarily call that the sort of out of the box solution, but there is the capability for that type of a solution to exist.
Tobias Macey: But I think that, broadly, then I agree with Esteban that even if there is an out of the box Open Source option and it's never going to fit everybody's use cases everybody's going to have their own.
Tobias Macey: Questions that they want to answer their own additional data sources that they want to use to enrich the analytics that they're building off of the open edx system.
Tobias Macey: You know they might have multiple systems that they want to compose together, and as long as we have a solution that is you know Compostable and swappable then.
Tobias Macey: You know, it leaves a lot of room for commercial players to be able to play in the ecosystem, so you know thinking in terms of having.
Tobias Macey: You know, saying an easy to use API is for a data extraction from the platform.
Tobias Macey: You know, for example, being able to query you know, in an API with a block ID and get the location and the course hierarchy is one thing, there.
Tobias Macey: And then you know, once you have the data extracted having a common set of having a set of DDT packages that act as common transform so if somebody has all their data.
Tobias Macey: In their warehouse they can pull in those DVD packages apply the transforms and then you have sort of the base set of these are the common elements that you want, and then you know, depending on how we've decided to.
Tobias Macey: standardize on the metrics layer having a set of predefined metrics that people can then tweak for their own use cases as long as all of those are composed, in a way that they work well together, but are also usable in isolation, then I think that is a sort of decent roadmap forward.
Miranda von Salis: That when, as you guys have been talking about this, I that was like one of the main things for me was that.
Miranda von Salis: Any metrics that we provide in the out of the box solution, need to be really clearly reproducible in the API and and, like any type of transformation packages or documentation or providing because.
Miranda von Salis: The thing people hate the most is when one dashboard says one definition of things and the other dashboards or something else.
Miranda von Salis: And they don't want to check any of them right and you're like trying to fudge the numbers enough to match what the out of the box one is saying Oh well, this says we have an enrollment of 151 in this webinar or whatever and you're like.
Miranda von Salis: And the other thing is that making it so that if it's an out of the box solution that comes with open edx, it is not something that.
Miranda von Salis: any type of open edx vendor who helps post and build sites can withhold drastically and up charge a whole lot for right if it comes with the instance, it should be like accessible and not something that they can like.
Miranda von Salis: slash access to like easily right, because if the whole you know what I mean, and I am very not that, like our vendors would do that to us because they're lovely people, but like you know we we operate through open craft they host our sites, we build them up all of our courses, but.
Miranda von Salis: Some of the functionality, you know they get is a lot will write with them and.
Miranda von Salis: The least amount we can do that, I mean, I know that, like.
Miranda von Salis: letting people make their livelihood is important but it's also like is that what we want to be encouraging.
Miranda von Salis: them to do right is so I don't know that's just a thought.
Edward Zarecor: I appreciate that, but I think one of the things.
I a.
lightning rod in open source projects yours is talking about money.
Tobias Macey: And i'll be honest with you, I think that having avenues to commercialization in the ecosystem are super important because it will help scale that ecosystem, so I think what we should be thinking about is.
Edward Zarecor: How can we get comfortable with kind of a plan for for having you know commercialization and much firms make money with open edx and configured back to the ecosystem.
But also giving people a reasonable experience for just the free version of the comments not i'm not a big fan of.
You know some of the models, where you get really, really useless version of it for free and then you can't really do anything without paying a lot for it, you get locked it's just like a it's a Trojan horse, to be honest with you, I don't want this project to have people a Trojan horses.
Miranda von Salis: But at the same time, I think we need to acknowledge for for the Community to persist for the Community to grow that there needs to be some avenues for companies and that's where there's any of this customization new dashboards integrating your your whatsapp data or whatever else.
Miranda von Salis: or even going from the two dashboards who's in it, what are they doing high level to the 10 dashboards estimates talking about, that is where I see that gap, but I I would be hesitant to not give at like the free folk anything right.
Miranda von Salis: Because you know analytics is where people like mana be are able to like monetizing really harness their data and I get that that's where the money is, but like what's what's the point if.
yeah.
Tobias Macey: To the point of sort of making analytics available out of the box by sort of hosting providers open crap, for example, I know that one of the things that.
Tobias Macey: has made that a bit of a non starter, with the inside stack is that it's quite expensive to run and complex to maintain and so by being able to simplify the architecture and make it.
Tobias Macey: Much sort of more scalable and cost effective, particularly for smaller installations, I think makes that an easier question to answer for those providers, where.
Tobias Macey: You know, rather than having to run a full emr stack and deal with all of that, if it's you know there's a standard set of api's.
Tobias Macey: You can put it into a pay as you go cloud data warehouse or something like that, with a default set of transforms that's a lot easier to get set up and it makes it.
Tobias Macey: Much more likely that these hosting providers are going to offer the analytic solution to their to their users, without adding extra charges, unless you want to then go beyond what is sort of the baseline out of the box.
Miranda von Salis: yeah i'm sorry yeah i'm talking about the baseline out of the box.
Tobias Macey: yeah absolutely.
Miranda von Salis: don't even know what they don't know right and it's going to whet people's appetites.
yep.
Sofiane Bebert: I just said in terms of the baseline out of the box what people feeling on what figure is doing, because I guess from my view, I felt figure would be bad just overview on on you have.
Sofiane Bebert: To be specific, course you can be basic progress.
Sofiane Bebert: By understand.
Sofiane Bebert: The project.
Sofiane Bebert: Maintaining and upgrading to the latest version.
Sofiane Bebert: And Is this something that we can help with or.
I wish that somebody from APP similar enjoy.
Just because of the timing, or because.
You know I can't speak to figures very directly, for you know good things and limitations of it but.
I think we need.
Evidence from out somewhere that they're advocating for it to us in that way and and maintaining.
Régis Behmo: I think that figures, this is a great product.
Régis Behmo: It does not do everything it only gets data from my sequel so it's not per se and full analytics stack it's really easy to install really easy to access.
Régis Behmo: it's not configurable we cannot create custom dashboards but their reports that come out of the box or good, so I think it's a.
Régis Behmo: it's a great it's a great.
Régis Behmo: Well, more than minimum viable product it's a.
Régis Behmo: Good baseline and and I think it's a shame that it's not maintained for the latest okay next releases.
Tobias Macey: yeah so I guess, one of the questions there as we're getting close to the end of the hour is sort of what are the barriers to maintenance, either in terms of.
Tobias Macey: their capacity for keeping up to date with it, you know, are there issues with the.
Tobias Macey: ways that the database schema is evolving that makes it challenging So what are the, what are the elements that makes it hard to upgrade across releases and then we can maybe start there and possibly see if we can.
Tobias Macey: Adopt that, as a community sort of supported project, you know, maybe make that something that is a.
Tobias Macey: sort of shepherded by this working group and just kind of spit balling there.
Andrés González: And I like to add another point, and maybe for the longer term, but it's about closing the information loop back to open index.
Andrés González: Because what everybody's doing is extracting data and showing it somewhere else, but it will be interesting, one thing is a maybe a product available they this information, this dashboard sort of whatever.
Andrés González: From the LM s a in the seminal Ms experience for the students and for the teacher and another thing will be interesting to to have a mechanism so so that the LM s can change the the experience or change the behavior based on that information that has come through analysis.
Andrés González: Maybe the what's called it in adaptive learning.
Andrés González: Maybe maybe change changing or changing the flow of the course or or changing the Romans based on some some analysis that comes from any analytic system that's can jokes in the loop on to the next.
Andrés González: I think nobody students something ladies.
Tobias Macey: Alright, so definitely lots more to discuss, but we are past the end of the hour, so I guess the final question Sophie i'm go ahead.
Sofiane Bebert: I just wanted to add just a tiny point around what interest head around feeding back, and I was just showing the data, maybe your microphone and could be a solution that could be easy and pluggable.
Andrés González: And solidly.
Sofiane Bebert: An interface for whatever solution, whether it's Karen whether it's panorama that's it okay.
Tobias Macey: So yeah I guess final question is.
Tobias Macey: Now that we've had our first meeting when's the next one, and what sort of.
Tobias Macey: rate of recurrence do we want to set for cadence for this group.
Tobias Macey: I know, with the btr group we do it every two weeks is that something that we want to at least trial for this group.
i'd be open to that.
Esteban Etcheverry: is reasonable it's.
Andrés González: Okay, for me.
Tobias Macey: And does this time work for everyone, or should we pick a different time.
Yes.
Tobias Macey: Okay.
Okay.
Edward Zarecor: Good talk I shared I created a section again called big crazy ideas and I added a bunch of stuff some based on what we were talking about just now.
Edward Zarecor: Take a look at that and if you are inspired to add, like other things that we should be thinking about for the long term roadmap of data, please add stuck there liberally.
Tobias Macey: So.
Tobias Macey: I will tend to to really set a recurring calendar event for this time every two weeks and I will try and broadcast that so that other folks who might want to take part can weigh in on if we should change the timeframes.
Alright sounds good.
Esteban Etcheverry: Okay.
Esteban Etcheverry: Great see everyone I.
Thanks.