18:00:30 <Lydia_WMDE> #startmeeting Wikidata office hour
18:00:30 <wm-labs-meetbot`> Meeting started Tue Nov 14 18:00:30 2017 UTC and is due to finish in 60 minutes.  The chair is Lydia_WMDE. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:30 <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:30 <wm-labs-meetbot`> The meeting name has been set to 'wikidata_office_hour'
18:00:40 <Lydia_WMDE> Hey folks :)
18:00:49 <Lydia_WMDE> how are you?
18:00:57 <Lydia_WMDE> who's here for the wikidata office hour?
18:01:07 <hweyl> i am!
18:01:09 * Lucas_WMDE is alive and well :)
18:01:19 <sjoerddebruin> I'm doing good and I'm here for that, surprisingly enough.
18:01:28 <Lydia_WMDE> \o/
18:01:59 <Lydia_WMDE> Ok then let's get the party started
18:02:25 <Lydia_WMDE> let's take a look at what happened over the last 3 months or so since the last office hour
18:02:59 <Lydia_WMDE> we've done quite some usability work
18:04:06 <Lydia_WMDE> specifically we've improved the search in the entity selector. that should now be considerably better. it can however still be tweaked more. let me know if you find cases where the ranking is still not good. This is the ranking for the results when you search for an item in the top-right search boy or when adding a new statement.
18:04:56 <Lydia_WMDE> we rolled out the constraints gadget that shows you when a statement is violating a constraint so this is easier to spot and hopefully fix than it was previously
18:05:20 <Lydia_WMDE> give it a try if you have not already and tell us what you think - specifically me and Lucas_WMDE
18:05:59 <Lydia_WMDE> we've also removed language fallback indicators for language fallbacks as these were pretty annoying when using wikidata in swiss german for example
18:06:20 <Lucas_WMDE> (for related languages)
18:06:50 <Lydia_WMDE> on top of that we've done a lot of talking to editors to get an overview of all the ways people are editing wikidata and where their biggest issues are
18:07:07 <Lydia_WMDE> Lucas_WMDE: heh right - sorry, forgot that important part :D
18:08:11 <Andra> I am here as well
18:08:24 <Lydia_WMDE> in this area the most important thing coming up is building out the constraint gadget and then turing it into proper code so we can turn it on for everyone as well as looking into shape expressions to see how they can be used to make the constraints checks even more powerful
18:08:33 * spinster says hi
18:08:52 <Lydia_WMDE> right then there is the big topic of structured data on commons
18:09:04 <spinster> :D
18:09:09 <VIGNERON> ;)
18:09:10 <Lydia_WMDE> we've continued the work on the underlying technology - specifically multi content revisions
18:09:20 <Lydia_WMDE> still a lot of work to do there but it is making progress
18:09:39 <Lydia_WMDE> and spinster will be doing an office hour next week to talk more about all the other work she and the team have been doing
18:09:56 <spinster> Yep, 21 Nov @ 18:00 UTC
18:10:02 <Lucas_WMDE> oh, cool! hadn’t heard about that
18:10:25 <Lydia_WMDE> go and attend if multimedia files and structured data are your thing :D
18:11:05 <Lydia_WMDE> alright next was the articleplaceholder. We've rolled it out to two more wikis: sqwiki and bnwiki.
18:11:24 <Lydia_WMDE> If you know a wiki that would like to have the articleplaceholder set up let me know.
18:11:33 <Lydia_WMDE> (unfortunately can't roll it out on large wikis yet)
18:12:16 <Lydia_WMDE> we've also put quite some work into support for all the other sister projects. for example there is now the wikidata concept monitor: http://wdcm.wmflabs.org/
18:12:35 <VIGNERON> \o/
18:12:36 <Lydia_WMDE> it helps you better understand how the different projects are using wikidata's data.
18:12:53 <Lydia_WMDE> if you find something interesting/weird/funny/... in there i'd love to hear about it
18:12:56 <sjoerddebruin> No font fallback :(
18:13:06 <Lydia_WMDE> sjoerddebruin: ?
18:13:30 <sjoerddebruin> There is no font stack in the CSS, just the font Liberation Sans that might not be on every computer.
18:13:42 <Lydia_WMDE> ah will let Goran know
18:14:13 <Lydia_WMDE> besides that we also did a lot of fire fighting to make change dispatching better
18:14:36 <Lydia_WMDE> this is what is responsible for notifying all the wikipedias and other projects about a change on wikidata that is relevant for them
18:15:24 <Lydia_WMDE> because of a lot of activity on wikidata we've been reaching the limit of what the current system could handle and have made some changes that buy us time but we'll need to look into more fundamental changes it seems
18:15:39 <sjoerddebruin> Things are doing well for 12 days now. :) https://grafana.wikimedia.org/dashboard/db/wikidata-dispatch?refresh=1m&orgId=1&from=now-14d&to=now
18:16:02 <bawolff> Is there any more thoughts about what those fundamental changes will be
18:16:13 <bawolff> (feel free to ignore that question if its out of scope for the meeting)
18:16:30 <Lydia_WMDE> as part of that we also started making the system more smart about only showing changes that are really relevant for that wikipedia or other project. before there was way too much in the watchlist and recent changes that wikipedia editors were not interested in
18:16:50 <Lydia_WMDE> unfortunately we need to roll that out slowly so we can monitor the performance
18:16:58 <Lydia_WMDE> so right now it is only on greek wikipedia and a few others
18:17:04 <Lucas_WMDE> sjoerddebruin: it has been 1️⃣2️⃣ days since the last dispatch incident :D
18:17:25 <Lydia_WMDE> bawolff: not really. we'll have to figure that out and probably have a discussion at the dev summit about it
18:17:45 <bawolff> ok. I'm very interested in what the long term plan will be
18:17:51 <Lydia_WMDE> *nod*
18:17:55 <Lydia_WMDE> me too :D
18:18:00 <bawolff> Its a complicated issue. Anyways, look forward to hearing about it at dev summit
18:18:08 <Lydia_WMDE> it is
18:18:16 <Andra> @Lydia Any indication on how much time was bought?
18:19:01 <Lydia_WMDE> Andra: it is really hard to say because we've been making all kinds of tweaks that might have small or ig impacts. also it is highly dependent on the number and kind of edits happening on wikidata
18:19:26 <Lydia_WMDE> I'd say months rather than years though
18:19:43 <Andra> :(
18:19:51 <Lydia_WMDE> jep indeed
18:20:04 <Lydia_WMDE> but then again: growing pains ;-)
18:20:15 <Andra> Indeed
18:20:34 <Lydia_WMDE> alright then on to the query service
18:20:52 <Lydia_WMDE> we've done more work on expereimenting with letting people click together queries without knowing sparql
18:21:08 <Lydia_WMDE> we've added support for showing the geoshapes from commons in the map visualisation
18:21:37 <Lydia_WMDE> and you can now easily get code snippets for your query to integrate it in your programs in all kinds of programming languages
18:22:18 <Andra> Yes I have seen that, NICE!
18:23:06 <Lydia_WMDE> Another thing we worked on was adding full URIs for external identifiers to our RDF exports. this means we'll hopefully soon show up in the Linked Open Data cloud visualisation. i'm looking forward to seeing how we'll be positioned there.
18:23:20 <Lydia_WMDE> http://lod-cloud.net/
18:23:34 <multichill> Good evening. Probably like if a nuke went off
18:23:42 <Lydia_WMDE> lol
18:24:54 <Lydia_WMDE> and then we also released a docker image for wikibase which hopefully makes it easier for people to start coding on wikibase and in the future also set up their own instance of wikibase.
18:25:31 <Lydia_WMDE> i'm really looking forward to the time when we'll have a nice ecosystem of many different wikibase installations out there all connected with wikidata but concentrating on their specialty data
18:26:27 <multichill> I like the improved search Lydia_WMDE! And the patch to solve pasting seems to be up for deployement too. Just wondering, when do we get strings in search? :-)
18:26:39 <Lydia_WMDE> multichill: \o/
18:26:53 <Lydia_WMDE> strings in search: that is a good question. SMalyshev is probably best to answer that
18:27:21 <Lydia_WMDE> which brings me to the last big development topic: lexicographical data and wiktionary
18:28:04 <Lydia_WMDE> that is what took most of the time over the last quarter and we now have a demo system at http://wikidata-lexeme.wmflabs.org of the current state. we've demoed that at Wikimania and have been working on the feedback since then.
18:28:24 <Lydia_WMDE> right now most of the edits you make there are not persistently stored. that is what we are tackling at the moment
18:29:03 <Lydia_WMDE> ok on to all the other important things that happened:
18:29:31 <Lydia_WMDE> there is now a page to help other wikimedia projects use the data in wikidata: https://www.wikidata.org/wiki/Wikidata:How_to_use_data_on_Wikimedia_projects
18:29:55 <Lydia_WMDE> and another one that adresses common questions/fears/... at https://www.wikidata.org/wiki/Wikidata:Wikidata_in_Wikimedia_projects
18:30:06 <Lydia_WMDE> both of these could use some more help if you have some spare time
18:30:43 <Lydia_WMDE> a bunch of new tools have been developed.
18:31:00 <Andra> Lydia: Maybe worth mentioning I am Currently brainstorming with Rhizome and the Finish name project on building such an ecosystem of collaborating Wikibases
18:31:06 <Lydia_WMDE> https://tools.wmflabs.org/wikidata-todo/sparql_rc.php lets you enter a sparql query and then gives you edits related to the result set.
18:31:22 <Lydia_WMDE> so you can for example query for all politicians in a country and then see what edits happened to their items
18:31:37 <Lydia_WMDE> Andra: awesome.
18:32:15 <Lydia_WMDE> here is another one: https://tools.wmflabs.org/wikidata-todo/wd_edit_stats.php
18:32:35 <Lydia_WMDE> it lets you enter a sparql query and a time range and then gives you statistics about the edits in that timeframe
18:32:48 <VIGNERON> \o/ already tried and adopted !
18:33:20 <Lydia_WMDE> https://angryloki.github.io/wikidata-constraint-violations/ shows you how the constraint violations for a specific property are developing over time. very useful to see the impact of a cleanup drive or monitor imports
18:33:44 <Andra> Lydia here is the Gdoc collecting the ideas: https://docs.google.com/document/d/1ax3um4KJWXK5TZHQjaphpXH-bBonLo9D1RfuLd6IWB4/edit?usp=sharing
18:33:49 <Lydia_WMDE> thanks!
18:34:16 <Lydia_WMDE> here is another helpful tool for cleanups: https://lucaswerkmeister.github.io/wikidata-ontology-explorer/
18:34:31 <Lydia_WMDE> that lets you visualize parts of the ontology and see where things are not right
18:34:38 <Andra> Also I can confirm that the docker image really works. I have deployed several since its release
18:34:44 <Lydia_WMDE> :D
18:34:45 <Lydia_WMDE> sweet
18:34:48 <Lucas_WMDE> \o/
18:35:20 <Lydia_WMDE> There is more good stuff in the birthday presents section on https://www.wikidata.org/wiki/Wikidata:Fifth_Birthday
18:35:30 <Lydia_WMDE> check it out
18:35:39 <Lydia_WMDE> oh and there is now an alexa skill for Wikidata :D https://www.amazon.de/metaphacts/dp/B0745KLCFX/
18:36:25 <multichill> I think some people first need to load the German skill before they can use that one Lydia
18:36:26 <bawolff> wow, wikidata is 5
18:36:31 <Lydia_WMDE> If you speak a language that is not well-represented on wikidata you should totally join the Wikidata flashmop: https://www.wikidata.org/wiki/Wikidata:Flashmob
18:36:35 <bawolff> It seems just yesterday it was announced
18:36:51 <Lydia_WMDE> multichill: i thoguht it also works for english now
18:36:51 <sjoerddebruin> Dutch is so under-represented...
18:36:59 <Lydia_WMDE> bawolff: :D i know right?
18:37:06 <Lydia_WMDE> growing up so fast!
18:37:09 <Lydia_WMDE> sjoerddebruin: :P
18:37:25 <Andra> Alexa is still mainly a english/german thing, isn't it?
18:37:43 <Lydia_WMDE> i'm not sure how many languages they cover atm
18:37:49 <Lydia_WMDE> but definitely english and german
18:38:01 <Lydia_WMDE> (individual skills might know less)
18:38:15 <Lydia_WMDE> speaking of 5 years old!
18:38:35 <Lydia_WMDE> we celebrated Wikidata's 5th birthday with a whole conference and I really really loved it
18:38:52 * VIGNERON loved it too
18:39:15 <Andra> Yes it was an awesome event
18:39:23 <Lydia_WMDE> https://www.wikidata.org/wiki/Wikidata:Fifth_Birthday has lots of nice texts, presents and more
18:39:25 <Lydia_WMDE> <3
18:39:35 <Lydia_WMDE> https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Documentation/Proceedings has documentation from the conference
18:39:45 <Lydia_WMDE> https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Documentation as well
18:39:54 <Lydia_WMDE> that has videos, slides, notes etc
18:40:00 <Andra> It was also the first conference I ever attended with really no Internet issues whatsoever
18:40:12 <Lydia_WMDE> *lol*
18:40:15 <Lydia_WMDE> very good
18:40:52 <Lydia_WMDE> that was actually one of our big worries but the venue was awesome and super helpful (Tagesspiegel, a local publishing house)
18:40:53 <spinster> I was wondering if there are plans to upload the video registrations to Commons?
18:41:00 <Lydia_WMDE> yes they will be
18:41:06 <spinster> :-)
18:41:23 <Lydia_WMDE> not sure when though (Léa's department and she's on vacation now so I guess when she is back=
18:41:39 <spinster> She surely deserves that vacation :D
18:41:44 <Lydia_WMDE> :D totally!
18:41:53 <Lydia_WMDE> until then you can get them from the chaos computer club
18:42:09 <Lydia_WMDE> who were fabulous in recording everything
18:43:45 <Lydia_WMDE> alright and my last part is blog posts and papers
18:44:11 <Lydia_WMDE> here are some that talk about using wikidata's data to build cool things:
18:44:12 <Lydia_WMDE> https://blog.wikimedia.de/2017/07/25/data-partnerships-with-wikidata-beatunes/
18:44:12 <Lydia_WMDE> https://blog.mapbox.com/exploring-the-world-with-wikidata-and-openstreetmap-30f1bfe954d3
18:44:12 <Lydia_WMDE> https://blog.wikimedia.de/2017/08/21/data-partnerships-in-wikidata-project-durchblick/
18:44:29 <Lydia_WMDE> and a paper about references on Wikidata: https://eprints.soton.ac.uk/412922/
18:44:59 <Lydia_WMDE> and one analyzing wikidata's language coverage: http://opensym.lero.ie/wp-content/uploads/2017/08/a14-kaffee.pdf
18:45:24 <Lydia_WMDE> alright. that's it for the reporting part of the office hour
18:45:35 <Lydia_WMDE> on to comments/questions/other topics you want to talk about :)
18:45:54 <Andra> Lydia: There was an issue with the Gdoc share setting its open for all to comment now
18:46:04 <Lydia_WMDE> thanks!
18:46:59 <hweyl> thank you for the updates, Lydia!
18:47:07 <Lydia_WMDE> you're welcome!
18:47:17 <sjoerddebruin> Glad to see some focus on stuff on Wikidata itself again, it was a lot of non-visible things lately.
18:47:53 <Lydia_WMDE> :)
18:48:44 <Andra> Can I ask a question on the Docker image here?
18:48:56 <Lydia_WMDE> all questions allowed :D
18:48:58 <Lydia_WMDE> so go ahead
18:49:14 * Lydia_WMDE wonders if addshore is around
18:49:22 <Envlh> thanks Lydia_WMDE! Like you, I'm looking forward for the Linked Open Data cloud visualisation update ("One Database to link them all" ^^)
18:49:29 <sjoerddebruin> Idle for like one minute ;)
18:49:31 <Andra> We are considering using the image as a test box for our bots.
18:49:36 <Lydia_WMDE> Envlh: :P exactly
18:49:52 <sjoerddebruin> I'm actually wondering if we should ask people to test more on test.wikidata.org and local images...
18:49:57 <sjoerddebruin> I still don't like the Wikidata sandbox :P
18:50:21 <Lydia_WMDE> sjoerddebruin: i guess it depends on what kind of tests
18:50:31 <Lydia_WMDE> large scale import tests: definitely!
18:50:34 <Andra> but this would require that the property number to be persistent, but now it seems that the p number are assigned automatically, can this be made manually?
18:50:40 <sjoerddebruin> If you want to do some tests with clients, yeah.
18:50:41 <Lydia_WMDE> bigger modeling: probably
18:50:56 <Andra> Sjoerd the issue with sandbox is that sometimes tests involves actual item and property numbers
18:51:06 <addshore> Lydia_WMDE: yes!
18:51:16 <Lydia_WMDE> addshore: see Andra's question please :)
18:51:22 <addshore> *reads up*
18:51:40 <addshore> "We are considering using the image as a test box for our bots." >> Sounds like a good idea
18:51:44 <sjoerddebruin> It is possible to exclude the sandbox items from queries, unless you've used some kind of service?
18:52:28 <addshore> "but this would require that the property number to be persistent, but now it seems that the p number are assigned automatically, can this be made manually?" >> As in you would require the same property on the docker container each time you use it?
18:52:28 <Andra> envlh/Lydia: It has been genewiki's slogan from the start (making wikidata the hub in the (biological) linked data cloud) ;)
18:52:36 <sjoerddebruin> Oh right:
18:52:37 <addshore> Or the same property Ids as on wikidata.org?
18:52:54 <Andra> addshore: the same property ids as on wikidata.org
18:53:04 <Andra> and even in some case on item numbers
18:53:10 <sjoerddebruin> When can we expect better property suggestions? :)
18:53:11 <Andra> but maybe that is pushing it
18:53:17 <addshore> So, that isnt really possible right now
18:54:00 <Andra> :(
18:54:18 <addshore> I did plan on incorporating https://github.com/filbertkm/WikibaseImport into the docker image, but then https://github.com/filbertkm/WikibaseImport/issues/14 is an issue that would still need fixing
18:54:20 * DanielK_WMDE_ missed the office hour
18:54:33 <addshore> Andra: you could however just have a mapping in some configuration file?
18:54:35 <Lydia_WMDE> sjoerddebruin: since we need to concentrate on the change propagation improvements i fear that is on the backburner for a while :/
18:54:54 <sjoerddebruin> I've seen some refactoring though, so it's something.
18:55:01 <Lydia_WMDE> yeah
18:55:41 <DanielK_WMDE_> addshore: WikibaseImport *maps* IDs, it doesn't preserve them.
18:56:04 <DanielK_WMDE_> addshore: to preserve, you can use a plain XML import.
18:56:27 <DanielK_WMDE_> why not just do that?
18:56:35 <addshore> DanielK_WMDE_: yup, although the plain XML dump then has issues with the hook up to the query service
18:56:52 <addshore> or, at least, it isn't as simple as it sounds / I would need to document more stuff around that area
18:56:56 <DanielK_WMDE_> addshore: it does? like what?
18:57:24 <addshore> With the simple playground docker images, you create items, they are in the query service via the updater, if you import a dump, that wont happen
18:57:25 <DanielK_WMDE_> yea, the xml import isn't safe, it produces a lot of red links, and it should only be done if no ids are created locally.
18:57:45 <DanielK_WMDE_> so, it's not for general use. that'S why it's deisabled per default
18:57:48 <DanielK_WMDE_> for initializing a testing environment, it should be fine
18:58:05 <addshore> WikibaseImport also has a task filed for json dumps https://github.com/filbertkm/WikibaseImport/issues/7 which would be great
18:58:20 <addshore> ad WikibaseImport creates recent changes entries, which means the updater picks up the changes
18:58:30 <DanielK_WMDE_> addshore: the query services needs to be re-loaded from a dump after na xml import...
18:58:36 <addshore> DanielK_WMDE_: yup
18:58:47 <DanielK_WMDE_> addshore: no, better: just provide an RDF dump along the XML dump. import both
18:59:09 <addshore> Andra: if you don't want to use the query service bits of the docker stuff you can try just loading XML dumps of the items / properties that you need
18:59:39 <Andra> addshore: actually the query service is essential in the test environmnet
18:59:58 <addshore> DanielK_WMDE_: well, the rdf dump still has to be munged, it can't just be thrown into blazegraph in 1 go.
19:00:30 <addshore> DanielK_WMDE_: you can dump zips of the all the mysql and blazegraph data, and send them to someone else to load into their own docker containers ;)
19:00:38 <Lydia_WMDE> Andra: are you editing all relevant items? then that would be enough to get them into the query service and you should be ok with an xml import
19:00:55 <Andra> Lydia: Indeed!
19:01:22 <addshore> Andra: It might be worth you filing a ticket under the #wikibase-containers project on phabricator to discuss further, as I know you are not the only person with this usecase.
19:02:05 <addshore> infact Andra https://phabricator.wikimedia.org/T180216
19:02:51 <Andra> addshore: Will do. We are also trying to see if we can get a workshop going on this: https://docs.google.com/document/d/1ax3um4KJWXK5TZHQjaphpXH-bBonLo9D1RfuLd6IWB4/edit
19:03:35 <Lydia_WMDE> sweet
19:03:44 <Lydia_WMDE> any other topics/questions?
19:04:15 <sjoerddebruin> Some help cleaning up https://www.wikidata.org/wiki/Wikidata:Database_reports/Constraint_violations/Mandatory_constraints/Violations would be great. :)
19:04:18 <hweyl> no more topics from me. thanks for the office hour!
19:05:01 <Lydia_WMDE> alright then let's wrap that up and all help sjoerddebruin clean up constraint violations :D
19:05:03 <spinster> I warmly invite everyone to next week's Structured Commons IRC office hour again! Nov 21, 18:00 UTC
19:05:09 <Lydia_WMDE> thanks everyone for coming!
19:05:40 <Andra> Thanks all
19:05:46 <VIGNERON> spinster: will it be on #wikimedia-office ?
19:05:57 <spinster> Yes
19:06:13 <VIGNERON> good, I'll be there ;)
19:06:14 * Lucas_WMDE waves goodbye
19:06:27 <Lydia_WMDE> #endmeeting