In T391815#10767798, @Olea wrote:

@Superraptor123, could you please provide more details for each case? If you find it appropiate, create their own new tickets.

Thanks!

About importing/exporting ontologies, I agree is a very important issue but, personally, today it's outside my skills. Maybe it deserves it's own ticket also, with all the technical details you could provide.

I would request the following additions: (1) an addition/extension for WikibaseQualityConstraint items/properties; (2) something akin to the SSSOM (https://mapping-commons.github.io/sssom/spec-intro/) for better mapping specifications, and (3) some basic SKOS properties (https://www.w3.org/TR/skos-reference/).

Confirmed! This works when adding to C:\Windows\System32\drivers\etc the following line: 127.0.0.1 wdqs-frontend.example.com. It does have to be opened as an administrator to edit, so anyone else who comes across this thread, see here (https://phabricator.wikimedia.org/T372177) for details. Thanks so much @roti_WMDE ! I'll close as solved.

Thanks! Added new ticket here: https://phabricator.wikimedia.org/T375386

Hey @roti_WMDE -- just following up on the above! Thanks so much!

Okay-- so I'm working through things and here's the what I've come against so far (noting this is just for WikibaseLexeme, as Kartographer and WikibaseQualityConstraints have more complicated requirements). So essentially I have a Python program that works like this (rough outline of the methods for use in deploy-3, not HEAD):

You are a genius-- okay, I got it up and running by running the following in my Python script (this copies a .gitattributes file into the build/Wikibase directory, builds the image, and then later in the script, docker compose up is run):

Triple-checking line endings in the file... they're all LF (\n) for ./build/Wikibase/entrypoint.sh (and every single other file in the repo-- I made sure by running dos2unix over the whole directory before running); so not sure where the issue is coming from:

Even completely uninstalling and reinstalling Docker does not help-- error still persists.

Umm... so stopping and downing the containers and deleting everything and attempting to run the vanilla Wikibase (without any of the WikibaseLexeme bits added; fresh cloned repo and all) is now giving a similar error and not running:

Running ./build.sh wikibase in Git Bash prints out a SHA code and then exits basically immediately. Running docker compose up in Git Bash results in the same series of errors. So not a CMD issue it would seem.

Checked all files for line endings and there is not \r anywhere in any of the files I've edited.

Okay so I'm testing the deploy-3 instructions now.

So exciting! I just finished testing your solution re: https://phabricator.wikimedia.org/T372177, so I'm super excited to get this up and running! I'm beginning testing on my machine now! You're the best!

Attempting solution now-- added the following line to hosts in C:\Windows\System32\drivers\etc:

Thanks so much for the thoughtful and detailed response! I'm not very familiar with port forwarding/proxy handling and I'm dealing with another issue with importing (munge.sh is giving me some strange errors, I'll document that elsewhere).

This is almost certainly related to Traefik rate limits (see here: https://github.com/wmde/wikibase-release-pipeline/blob/main/deploy/README.md, under "Removing Wikibase Suite Completely with all its Data"). Because of this I'm closing for now, but I do think this should be mentioned earlier in documentation, as it seems to be an error multiple users are having.

Got it back up and running (I think this was a certificates error, but it failed silently, not sure what's wrong), see: https://github.com/wmde/wikibase-release-pipeline/blob/main/deploy/README.md (under the section "Removing Wikibase Suite Completely with all its Data")

I thought maybe the issue was something to do with line 194 in docker-compose.yml, so I waited a couple days to re-spawn the system (until just now), but it is still not working:

The complete reinstallation is not working anymore; documented this here: https://phabricator.wikimedia.org/T373138. The whole pipeline now seems to be broken and I can't get it up at all.

I will note this: after making any changes to the Dockerfile, even reverting them causes:

As a simple test I:

Well, solved it. It's definitely not a wikibase-release-pipeline issue, but an issue with Docker. I've posted this issue over in the Docker support area.

Attempted to disable the WSL 2 based engine (per here: https://github.com/docker/for-win/issues/4495), did not work either, same issue.

Attempted updating Docker, running Docker as administrator, and restarting Docker, no dice.

@roti_WMDE so you're absolutely right about me missing .env in the last command; but even with it I'm still getting dependency failed to start: container wbs-deploy-wikibase-1 is unhealthy.

Found this thread: https://github.com/docker/for-win/issues/13611. Suggested maybe that the issue was having the Dockerfile open in Visual Studio Code (weird, but hey stranger things have happened). Attempted the fix and still same issue with wbs-deploy-wikibase-1.

I should also note that I had been able to get the Wikibase up successfully before today using the same commands, but I had re-cloned the GitHub repo today. I have not tested rolling back the GitHub repo but will attempt tomorrow.

Okay! Thanks so much.

Ok-- so nothing is needed at all with etc/hosts. You just have to change WIKIBASE_PUBLIC_HOST in .env to localhost. That's it! Closing this, but I still think the documentation should be updated to reflect this.

Confirming it also works for lexemes! It does require search indexing first though (starting from the update.php line and moving forward):

Confirming that this works for items and properties! It seems like lexemes may be a bit more involved though-- working on that now and will post final scripts!

Apologies-- I am just now returning to work from holiday and I plan to attempt this solution in the next few hours. I will report back on progress-- thanks so much!

Has there been any movement on this? Thanks so much!

Even though this is the case, this behavior should be documented in the Wikibase API docs.

While testing with the smaller dump from https://furry.wikibase.cloud/ I've discovered the following behaviors that I think are interesting (but not sure if relevant):

@roti_WMDE just updating so all info is in one place; discussed this briefly with Deniz at the Wikibase Community User Group meeting last week. Having posted on StackOverflow (https://stackoverflow.com/questions/78663492/creating-new-items-after-importing-a-wikibase-xml-dump) and on GitHub (https://github.com/mediawiki-client-tools/mediawiki-dump-generator/issues/249), I have tried all feedback received to no avail (same issues). Let me know if there's anything I can test or get together on my end! Thank you so much for your help!

Just checked docker volume ls. Output when down is:

Thanks so much for reaching out! I'll run another test shortly for the MariaDB just in case!

I think this may be an issue with the XML dump created by MediaWiki Dump Generator, so I also opened a ticket there, just in case (https://github.com/mediawiki-client-tools/mediawiki-dump-generator/issues/249).

I thought maybe wikibase-lexeme was the issue since the SQL script didn't rebuild the counters for lexemes, so I added the following line: REPLACE INTO /*_*/wb_id_counters VALUE((SELECT COALESCE(MAX(CAST(SUBSTRING(page_title`, 2) AS UNSIGNED)), 0) FROM page WHERE page_namespace = 146), 'wikibase-lexeme');`. The file outputs the following:

Maybe it's possible that I'm not shutting down correctly when instantiating the Docker containers? I've been running docker-compose -f docker-compose.yml -f docker-compose.extra.yml down --volumes --remove-orphans between tests... Is there something else I should run? Maybe the mysql database persists somehow and is messing with the success of the SQL script? Is there a different or better way to cleanly shutdown the Docker components?

Just in case, also posting output of first script here:

Managed to get together (essentially) a full log of everything in the second script above, just in case it's helpful. Unfortunately, it's too large to upload the whole thing, so here are the highlights:

Output of initSiteStats.php:

Adding output of SQL script (php /var/www/html/maintenance/sql.php /var/www/html/maintenance/rebuildWikibaseIdCounters.sql), as mentioned on Telegram:

Thirded-- can confirm autocomplete in the search bar works, but general search still gives "An error has occurred while searching: We could not complete your search due to a temporary problem. Please try again later."

Can say in relationship to my instance that the SPARQL queries stopped updating between 21:16 on 26 March 2024‎ and 13:42 on 27 March 2024.

Seconding-- I have been experiencing the same issue consistently since 24 March 2024.

Just adding my specific use case for my instance here; the reason I think this would be useful for me specifically is so that my instance can be made into an ontology posted quarterly or yearly to GitHub as a TTL file. This is because my instance is, in many ways, a successor to an ontology I created for my thesis that is continually updated. Being able to pipeline this out in a semi-automated manner 1-4 times a year would be ideal, since BioPortal, Ontobee, etc. automatically take and curate from GitHub.

It appears on the most recent version of the site that this feature is now implemented! Closing out the ticket now.

Yeah, to be fair the last time I tested the service was maybe more than a couple months ago, and it seemed to be working fine (even updating really quickly), but now nothing is showing up at all-- a reload would be amazing if that's possible!

Thanks so much for the details and for fixing! Really appreciate it!

@Tarrow No worries! Thank you so much for helping work on this! Unfortunately I will note that directly referencing Property ID is also not working at the present time.

Was redirected from Wikibase.Cloud / WBStack Telegram to report search issues since platform update from MediaWiki 1.37 to 1.38. Currently my instance (https://lgbtdb.wikibase.cloud/) is getting "An error has occurred while searching: We could not complete your search due to a temporary problem. Please try again later." for all searches since the update. As well, it is not possible to create new statements, as no entities appear in that search feature either (i.e. typing any property results in a "No match was found" error). Is there any way I can help with diagnosing or addressing the issue? I can also provide more information or feedback if needed. Thank you all so much!

@Argahsuknesib I got frustrated and abandoned the platform for about a year sadly;

Okay, so I think I've figured out how to do this, based on a hint in this article here. You actually need to use a proxy server. This will be different for people running different servers, but since mine is Apache2, I was able to find a relatively simple guide here. What I did was set up a proxy server on port 9000, which redirects to port 9999 (where Blazegraph is located). After doing this, the Wikidata Query Service GUI actually returns results! However, the links are a bit wonky still, so some formatting needs to be done.

After finally figuring out the other error, as documented here, I'm back at this error again, with the CORS issue. Any thoughts?

So it appears that there was something, somewhere which was still connected to the previous Maven instance, and resetting the whole server worked in order to get Blazegraph itself running... however, I'm just back at square one, being unable to set CORS so that the Wikidata Query GUI can access the Wikidata Query Service. It's the same as what I've documented here, so I'll return over to that thread.

I haven't found a solution. I have tried rebuilding things from scratch multiple times encountering different, seemingly unrelated, errors each time. I think it's still useful for this to be considered, however, so a solution should still be sought after. For the time being, I am focused on one of those different errors which I've documented here.

This bug continues to be pervasive... just in case, I've attempted several tests with mvn install, mvn clean install, and mvn package. mvn package runs without error, but the other two each produce the following error:

So I've pulled apart the logs of mvn install clean, mvn install, and mvn package in the ./wikidata-query-rdf directory, and it turns out that it is deleting the jetty-servlets-9.4.12.v20180830.jar in ./wikidata-query-rdf/war/target/blazegraph-service-0.3.111-SNAPSHOT/WEB-INF/lib, as indicated with the following trace output:

UPDATE 1: This solution was posted for a Jetty version before Jetty 9 which is used in the version of wikidata-query-rdf that I am using, so it was not useful.

Update 1: This is probably the solution, but I still haven't gotten it to work (as of the most recent comment from 15 May 2022). It is certainly the closest to what is mentioned in the official documentation for Jetty 9, which is what I'm currently following (although without success as of yet).

I have attempted to change each web.xml to include the following, but it still isn't functional (same errors):

Additional information: so when I copy and paste the URL http://localhost:9999/bigdata/sparql?query=SELECT%20*%20where%20%7B%3Fa%20%3Fb%20%3Fc%7D%20LIMIT%2010 into the browser, it spits out valid XML, it just doesn't when the request is made from the GUI, which results in a net::ERR_FAILED 200, which I believe is a CORS issue.

For the most part, this is now resolved. Because I was tunneling into the instance remotely, I needed to use:

@Addshore I'm confused by your link (https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_options.html#conceptBaseUri); so if I want to change the URIs from the default http://wikibase.svc/entity/Q39 to localhost:8181/entity/Q39, I do this by adding something to LocalSettings.php?

This is my current YAML (it still isn't working):

Okay, wow I'm dumb. Running this inside the wikibase WDQS docker container made queries work:

@Addshore so sorry to keep bothering, I'm just trying to make sure I'm following everything correctly.

@Addshore yeah so I was so knee deep in errors, I deleted everything with my first try and attempted to use this Git directory instead: https://github.com/wmde/wikibase-docker/blob/master/README-compose.md

@Addshore thank you so much for the response. I ended up deleting everything and attempted to follow a different set of Wikibase install instructions (except now post-load runUpdate.sh just runs forever (even though there is very little that I added to the database); localhost:8282 queries are stall forever.

@Addshore thank you so much for responding!

@Louperivois I've run into the issue of docker_compose_files_wdqs-updater_1 crashing and restarting constantly as well.