Change 597864 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/tools/cli@master] WIP DNM port mwdd v1 to this go repo

https://gerrit.wikimedia.org/r/597864

If I could propose a slightly different take on this (although I might be missing context, sorry for missing the meeting yesterday):

MDD provides, as a default, database replication, PhpMyAdmin, Grafana, Redis, and Nginx proxy, in a (at least somewhat) opinionated way. While I would like to see as an end goal that mw cli could be used to create this type of setup, I don't think I would agree that it should be the default setup. I would love to instead provide an interactive setup tool that can provide generate the relevant docker-compose.override.yml configuration (or reading from multiple pre-defined docker-compose.yml files) depending on what the user wants: pick the PHP version, database engine, if there should be db replication, if there should be redis, etc. If we have a really good configuration setup tool then we can make it easier to for users to mix and match according to what they need. This can also allow us to better support development on lower resourced machines.

I agree that it shouldn't be the default setup too, in the v1 branch of mwdd this is reduced to proxy, dns, mediawiki and db.

depending on what the user wants: pick the PHP version, database engine, if there should be db replication, if there should be redis, etc.

This is again the goal of mwdd v1 and also what is in the WIP go patch. You can for example:

• mw srv create mw # this will create a proxy, dns, mediawiki and db currently (db could be removed if people really want sqlite, and that should be easily possible)
• mw srv create redis # creates redis
• mw src create phpmy admin # creates phpmyadmin

So the system is as you describe, a composition of various different separate compose yml files, rather than a default bloated system with no flex.

One of the things in the call that was discussed is now close or far apart this more advanced setup should be vs not using the CLI tool and just using the docker-compose file in core.
The general consensus was that this should not at all replace the docker-compose file in core, and going that route should be separate.

In T253313#6158027, @Addshore wrote:

I agree that it shouldn't be the default setup too, in the v1 branch of mwdd this is reduced to proxy, dns, mediawiki and db.

depending on what the user wants: pick the PHP version, database engine, if there should be db replication, if there should be redis, etc.

This is again the goal of mwdd v1 and also what is in the WIP go patch. You can for example:

• mw srv create mw # this will create a proxy, dns, mediawiki and db currently (db could be removed if people really want sqlite, and that should be easily possible)
• mw srv create redis # creates redis
• mw src create phpmy admin # creates phpmyadmin

So the system is as you describe, a composition of various different separate compose yml files, rather than a default bloated system with no flex.

One of the things in the call that was discussed is now close or far apart this more advanced setup should be vs not using the CLI tool and just using the docker-compose file in core.
The general consensus was that this should not at all replace the docker-compose file in core, and going that route should be separate.

Cool, thanks for clarifying.

• mw srv create mw # this will create a proxy, dns, mediawiki and db currently (db could be removed if people really want sqlite, and that should be easily possible)

The proxy and DNS are needed for facilitating inter-container network requests?

In T253313#6158093, @kostajh wrote:

The proxy and DNS are needed for facilitating inter-container network requests?

So the proxy is to make the user facing interaction nicer, so that the whole environment can be exposed with a single port for HTTP rather than needing to keep a custom port mapping up to date and thus remember .
This also makes all of the dev environments more similar and require less wiring.
Services currently then choose the VHOST that they want to expose themself via.
This does require updating the hosts file of a system to make it all work nicely, but that solution has worked fairly well with mwdd so far and can be made even easier with a better CLI tool
Whenever a new service is added that has a new host that is not in your hosts file the tool can prompt you to add it or add it for you.

As for the DNS, this was mainly added so that mediawiki an other services internal to the docker network can easily use the same vhosts that are in the proxy, without needing to go via the host network or have an extra bridge.
Some of the main use cases of this are for example wikibase requiring API communication with a second mediawiki install. There are of course other things that also need this.
This is another point in favour of using a proxy however, as again, doing this only with the provided "service" dns entries provided by docker-composer and ports would be complex.

In PS9 of https://gerrit.wikimedia.org/r/c/597864 the bulk of the mwdd v1 system works to some degree, so you can already run it, setup mediawiki and play around
Still a bit to go, but the main picture is now there..

Right now I continued work on the patches that I started a few days ago, which also alter the current "docker" command.

Would everyone prefer if I leave that alone for now and alter this patch to instead introduce a separate "mwdd" command initially?

Would everyone prefer if I leave that alone for now and alter this patch to instead introduce a separate "mwdd" command initially?

It's fine with me if we add subcommands to docker, e.g. mw docker {cmd}. Ideally mw docker would be the overview / help command that lists all the subcommands.

In T253313#6160935, @kostajh wrote:

Would everyone prefer if I leave that alone for now and alter this patch to instead introduce a separate "mwdd" command initially?

It's fine with me if we add subcommands to docker, e.g. mw docker {cmd}. Ideally mw docker would be the overview / help command that lists all the subcommands.

I guess the plan would then be to have some commands other than docker at some point? :)
Just checking as commands like this are starting to get a little long mwcli docker service mediawiki start.
But perhaps if interacting with docker services can live under docker then I could remove the service bit

I guess the plan would then be to have some commands other than docker at some point? :)

Right, I think for example the medium/heavyweight local environment (local-charts) may move into this project (e.g. mw local-charts ...) and I could also see adding e.g. a download utility for mw get SomeExtension or extension management mw enable/disable SomeExtension.

I would keep the service part of the subcommand, but for starting up the entire local stack (which may have multiple services) we could probably keep the existing mw docker start and mw docker stop commands, and those can be adjusted internally to call the relevant mw docker service {serviceName} start/stop commands.

In T253313#6161708, @kostajh wrote:

I guess the plan would then be to have some commands other than docker at some point? :)

Right, I think for example the medium/heavyweight local environment (local-charts) may move into this project (e.g. mw local-charts ...) and I could also see adding e.g. a download utility for mw get SomeExtension or extension management mw enable/disable SomeExtension.

Gotcha

I would keep the service part of the subcommand, but for starting up the entire local stack (which may have multiple services) we could probably keep the existing mw docker start and mw docker stop commands, and those can be adjusted internally to call the relevant mw docker service {serviceName} start/stop commands.

Would you want to keep the cli being able to interact directly with the thiner mediawiki-docker docker-compose file in core?

Would you want to keep the cli being able to interact directly with the thiner mediawiki-docker docker-compose file in core?

I think that would be nice, but we could also reconsider the presence of the docker-compose.yml file in core once the mw CLI is ready for use. Is the main problem the lack of compatibility between the mediawiki-docker-dev version 2 compose files and core's version 3 file? If so I think we should do the (unfortunately, tedious) work of converting the docker-dev compose files to version 3.

In T253313#6164456, @kostajh wrote:

Would you want to keep the cli being able to interact directly with the thiner mediawiki-docker docker-compose file in core?

I think that would be nice, but we could also reconsider the presence of the docker-compose.yml file in core once the mw CLI is ready for use. Is the main problem the lack of compatibility between the mediawiki-docker-dev version 2 compose files and core's version 3 file? If so I think we should do the (unfortunately, tedious) work of converting the docker-dev compose files to version 3.

That is indeed a major factor.
We should also confirm if, for example, 2.2 files work with 2.3 too, or even if 3.0 works with 3.1/
If different docker-compose files need to be the exact same version then having any sort of distributed collection of files won't really work.

There are also some other points, such as looking the services in the file up to internal DNS etc, however I think that could be programmed around by the CLI adding these bits to files that it pulls in.

We should also confirm if, for example, 2.2 files work with 2.3 too, or even if 3.0 works with 3.1/

If one file declares its version as 2.2 and the other says 2.3, docker-compose will refuse to process them together.

If different docker-compose files need to be the exact same version then having any sort of distributed collection of files won't really work.

Right. I think we should avoid this, until / unless there's a compelling case for supporting it.

Change 615478 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/tools/cli@master] Add 'docker ps' command

https://gerrit.wikimedia.org/r/615478

Change 615478 merged by jenkins-bot:
[mediawiki/tools/cli@master] Add 'docker status' command

https://gerrit.wikimedia.org/r/615478

In T253313#6158412, @Addshore wrote:

In PS9 of https://gerrit.wikimedia.org/r/c/597864 the bulk of the mwdd v1 system works to some degree, so you can already run it, setup mediawiki and play around
Still a bit to go, but the main picture is now there..

Right now I continued work on the patches that I started a few days ago, which also alter the current "docker" command.

Would everyone prefer if I leave that alone for now and alter this patch to instead introduce a separate "mwdd" command initially?

@Addshore @jeena I commented on the patch; I'm not so sure about adding a separate mwdd subcommand. IMHO it will be kind of confusing for the end user. Could we consider using a configuration file and a command like mw docker init that would provide a preconfigured development environment like mediawiki-docker-dev defines?

I also commented on the gerrit change, but perhaps the conversation should happen there.

So the main problem here is that mediawiki-docker (the docker-compose file provided in mediawiki.git) and what mediawiki-docker-dev does are fundamentally different things.

mediawiki-docker and thus the "docker" command in this cli is a wrapper around docker-compose, automating some of the things that you need to do to manage the docker-compose file and setup.

mediawiki-docker-dev is a more complete development environment that needs control over things.

One such issue highlighted in the past is that mwdd will come with docker-compose snippets to be included as part of the docker-compose project that is used for the dev environment.
In the way that mediawiki-docker is implemented, this would mean keeping versions of docker-compose files in sync that live in different places, as well as having certain mediawiki cli versions for different versions of mediawiki.
That works for the scope of mediawiki-docker, but will not work with the extra things mediawiki-docker-dev wants to bring.

---

I think this is the point we need to decide if mwdd is going to live in this cli tool at all? or if this tool should only focus on the mediawiki-docker docker-compose based setup.
I'm more than happy to develop a go cli tool along side this one for mwdd.
My goal here is to sunset the existing old mediawiki-docker-dev setup.
This setup is still used by most WMDE engineers and I'm sure it's used by quite a few other engineers too.

In T253313#6773720, @Addshore wrote:

I also commented on the gerrit change, but perhaps the conversation should happen there.

So the main problem here is that mediawiki-docker (the docker-compose file provided in mediawiki.git) and what mediawiki-docker-dev does are fundamentally different things.

mediawiki-docker and thus the "docker" command in this cli is a wrapper around docker-compose, automating some of the things that you need to do to manage the docker-compose file and setup.

mediawiki-docker-dev is a more complete development environment that needs control over things.

Hmm, I don't see them as fundamentally different, I believe both are tools aiming to provide abstractions for developers to allow them to do complex stuff (setting up and configuring services) with minimal fuss, in reproducible and shareable ways. The "docker" command name could perhaps be renamed, because what it is really doing is building up an environment where other services can be added on.

One such issue highlighted in the past is that mwdd will come with docker-compose snippets to be included as part of the docker-compose project that is used for the dev environment.
In the way that mediawiki-docker is implemented, this would mean keeping versions of docker-compose files in sync that live in different places, as well as having certain mediawiki cli versions for different versions of mediawiki.
That works for the scope of mediawiki-docker, but will not work with the extra things mediawiki-docker-dev wants to bring.

I think this would be worth discussing and reaching some agreement on, because yeah I could see how this point (should the configuration live in lots of different repos or in a single one) result in two different tools making more sense.

---

I think this is the point we need to decide if mwdd is going to live in this cli tool at all? or if this tool should only focus on the mediawiki-docker docker-compose based setup.
I'm more than happy to develop a go cli tool along side this one for mwdd.
My goal here is to sunset the existing old mediawiki-docker-dev setup.
This setup is still used by most WMDE engineers and I'm sure it's used by quite a few other engineers too.

overall I'd love to see convergence on a single tool / approach rather than more fragmentation in the local environment landscape for MediaWiki. To give just one example, performance optimization on Docker for Mac is a complex issue and it would be unfortunate to be working on addressing that in two separate tools.

I think this would be worth discussing and reaching some agreement on, because yeah I could see how this point (should the configuration live in lots of different repos or in a single one) result in two different tools making more sense.

Is discussion on this task good enough?
For me the pros/cons are:

Distributing configuration per service repo
Pros:

• Easier for devs to "own" the configuration for their project and update it
• Devs don't really need to understand the inner workings of the cli
• Less need to rely on others to make changes to local dev projects to get their project running in it
• cli project possibly has less code/is less complex
• It could be used without the cli as a stand-alone thing(?)

Cons:

• Need to find a way to unify the docker-compose versions
• Files being distributed across repos possibly makes it difficult to understand how all the pieces fit together

Snippets for each project in cli repo
Pros:

• Easier to make changes that apply to all or many projects
• Possibly more convenient for testing the cli
• ... other stuff I haven't thought of

Cons:

• Cli repo could get difficult to manage due to complexity
• Yet another place devs have to make changes to to get their project running/extra cognitive load

Hmm, I don't see them as fundamentally different, I believe both are tools aiming to provide abstractions for developers to allow them to do complex stuff (setting up and configuring services) with minimal fuss, in reproducible and shareable ways. The "docker" command name could perhaps be renamed, because what it is really doing is building up an environment where other services can be added on.

I guess the key here is they have a totally different implementation that ends up making them not so compatible with each other.

I think there are a few more pros and cons of each side that I can think of..

Distributing configuration per service repo

Pros:

Cons:

Development environment parts are versioned with the services being used (this is an issue if say I want to write something for REL1_32 for example, it might not have a dev env, or might not have the right or matching version)

Snippets for each project in cli repo

Pros:

Easier to control the "quality" of the development environment, and test supported configurations
Can provide extra development environment functionality centrally such as reverse proxies to make the dev environment "nicer"
Can more easily share resources that may want to be shared, redis, memcached, sql etc (alternative is each repo that needs a service also has to define it's own)
On the same line of convenience around services like this, shortcuts like "mwdd redis:cli" can be provided, I don't see how this can so easily happen with the distributed route

Cons:

Easier for devs to "own" the configuration for their project and update it

I don't think there should be a whole lot of "configuration" going on here.
I think if services want to be included in the setup, they should provide a docker image that can be used, if there is configuration that needs to happen this should happen through env vars or mounted files in that service.
Defaults should also be useable.
It'd be interesting for me to see how this would pan out with trying to include an additional service in the v1 branch of mwdd at https://github.com/addshore/mediawiki-docker-dev/tree/v1 which is very similar to the apporach I was going for in go / mwcli.

At the end of the day this is primarily a mediawiki dev environment in my eyes, and that will always be the central core.

In T253313#6782368, @Addshore wrote:

Hmm, I don't see them as fundamentally different, I believe both are tools aiming to provide abstractions for developers to allow them to do complex stuff (setting up and configuring services) with minimal fuss, in reproducible and shareable ways. The "docker" command name could perhaps be renamed, because what it is really doing is building up an environment where other services can be added on.

I guess the key here is they have a totally different implementation that ends up making them not so compatible with each other.

I think there are a few more pros and cons of each side that I can think of..

---

Distributing configuration per service repo

Pros:

Cons:

Development environment parts are versioned with the services being used (this is an issue if say I want to write something for REL1_32 for example, it might not have a dev env, or might not have the right or matching version)

Snippets for each project in cli repo

Pros:

Easier to control the "quality" of the development environment, and test supported configurations
Can provide extra development environment functionality centrally such as reverse proxies to make the dev environment "nicer"
Can more easily share resources that may want to be shared, redis, memcached, sql etc (alternative is each repo that needs a service also has to define it's own)
On the same line of convenience around services like this, shortcuts like "mwdd redis:cli" can be provided, I don't see how this can so easily happen with the distributed route

I think this also makes it easier for developers to contribute to a single repo.

For usage, if the config is distributed across multiple repos the developer story for ensuring compatibility when changes are made sounds complicated.

Cons:

---

Easier for devs to "own" the configuration for their project and update it

I don't think there should be a whole lot of "configuration" going on here.
I think if services want to be included in the setup, they should provide a docker image that can be used, if there is configuration that needs to happen this should happen through env vars or mounted files in that service.

What would this look like for non Wikimedia services like PHP-FPM, MariaDB, httpd, redis, elastic search, etc? We have images for most of those in releng/dev-images so I guess we would require that any service (e.g. mariadb, which doesn't have an image in releng/dev-images repo yet) must have a WMF maintained image?

Defaults should also be useable.
It'd be interesting for me to see how this would pan out with trying to include an additional service in the v1 branch of mwdd at https://github.com/addshore/mediawiki-docker-dev/tree/v1 which is very similar to the apporach I was going for in go / mwcli.

At the end of the day this is primarily a mediawiki dev environment in my eyes, and that will always be the central core.

What would this look like for non Wikimedia services like PHP-FPM, MariaDB, httpd, redis, elastic search, etc? We have images for most of those in releng/dev-images so I guess we would require that any service (e.g. mariadb, which doesn't have an image in releng/dev-images repo yet) must have a WMF maintained image?

Yes

This is one shortcut that has been taken in the past with mwdd, in the way that it just uses images from docker hub.

It should be fairly easy to provide a mix of images if we can't make and maintains all the ones we want in releng/dev-images for example.

Infact, only using images from releng/dev-images would already slow down the port of mwdd to go :/

Now that various refactorings have happened in mwcli this gets a bit easier from here on.
I'm also doing development on Github now and pushing things when they are ready to gerrit changes for review, then I can take advantage of github actions.

The current set of things is at https://gerrit.wikimedia.org/r/q/project:mediawiki/tools/cli+status:open+author:addshorewiki%2540gmail.com+-has:draft+-WIP (and I'll be using the topic "mwdd" with all mwdd specific patches).

Current patches allow for create and destroy in the same style as mwdd.
Next will be 1) fetching mediawiki code, 2) composer installability 3) single mysql service 4) mediawiki install command

https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/682240 gets the code to a point where you can install mediawiki on either sqlite or mysql.
Running mysql is an optional service.

There are still a variety of "nice" things that I expect people using the existing mwdd would want added before thinking of switching over:

• Easy altering of the hosts file if needed
• Easy getting a shell inside the mediawiki container
• Easy phpunit running?
• Documentation for the existing functionality
• Somewhere to download the binary from (and changelog etc)

And secondary probably:

• Easy viewing of logs from mediawiki
• Some web based mysql viewer (adminer or phpmyadmin)
• Easy running of mw maint scripts?
• mysql replica

Change 615580 had a related patch set uploaded (by Addshore; author: Addshore):

[mediawiki/tools/cli@master] mwdd: Installing with sqlite

https://gerrit.wikimedia.org/r/615580

Change 682240 had a related patch set uploaded (by Addshore; author: Addshore):

[mediawiki/tools/cli@master] mwdd: Enable installing with mariadb

https://gerrit.wikimedia.org/r/682240

Change 597864 abandoned by Addshore:

[mediawiki/tools/cli@master] WIP DNM port mwdd v1 to this go repo

Reason:

See the other patches in https://phabricator.wikimedia.org/T253313

https://gerrit.wikimedia.org/r/597864

Change 682310 had a related patch set uploaded (by Addshore; author: Addshore):

[mediawiki/tools/cli@master] mwdd: Implement nice destroy commands for individual services

https://gerrit.wikimedia.org/r/682310

Change 682311 had a related patch set uploaded (by Addshore; author: Addshore):

[mediawiki/tools/cli@master] mwdd: Implement mediawiki & mysql suspend & resume commands

https://gerrit.wikimedia.org/r/682311

Also:
https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/682309
https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/683731
https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/683624
https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/683736

The best place to look is the topic https://gerrit.wikimedia.org/r/q/topic:%22mwdd%22+(status:open)

In terms of the checklist from a few days ago, ignoring docs and somewhere to download the built thing from, mwdd in its most basic form is now probably missing....

• Initial setup
  • Some nice wizard? https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687144
  • Fetching core and Vector https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687144
  • composer install https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687302
• Cleanup of things up for review already
  • Command examples and details https://phabricator.wikimedia.org/T282338
  • mediawiki exec as current user https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687017
  • mediawiki phpunit as current user https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687305
• Some additional things would be nice
  • postgres https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687178
  • redis https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/683789
  • adminer https://gerrit.wikimedia.org/r/c/mediawiki/tools/cli/+/687183
  • fresh integration https://phabricator.wikimedia.org/T282331
  • mysql replica https://phabricator.wikimedia.org/T282332
  • statsd https://phabricator.wikimedia.org/T282333
  • quibble https://phabricator.wikimedia.org/T282334
  • "raw" execution of docker-compose commands https://phabricator.wikimedia.org/T282335
  • make use of composer cache from host https://phabricator.wikimedia.org/T282336
  • Altering hosts file if desired https://phabricator.wikimedia.org/T282337

Some things were done:

• Easy getting a shell inside the mediawiki container
• Easy phpunit running?

I'm moving to start adding new things and fixes to the existing patch chain to phabricator under their own tickets on the mwcli board, using the prefix [mwcli mwdd]

The chain of patches for review for this ticket are at https://gerrit.wikimedia.org/r/q/topic:%22mwdd%22+(status:open)
And probably once the patches that are currently up are merged we could close this "monster" ticket and start using streamlined tickets.

Then the next goal would be to get a build out for people to use etc!!!

I have gone ahead and created tickets for all the TODOs in code, TODOs in commit messages, ideas from comments in code review and on phab & emails that I could find.

I think that wraps up the initial port

T273672: [mwcli docker] Mediawiki-Docker cli broken success message
T282329: [mwcli mwdd] Do not use "latest" images by default
T282334: [mwcli mwdd] quibble integration
T282337: [mwcli mwdd] Automatic hosts file altering
T282339: [mwcli] Create a docs site
T282355: [mwcli] check if git is installed and complain nicely if it is not before running git commands
T282353: [mwcli mwdd] Provide a command that can get more than just core & Vector
T282352: [mwcli mwdd] Check that the suggested default port to use for mwdd can be used
T282351: [mwcli mwdd] If the user is picking a MediaWiki directory & we can detect mediawiki, then suggest that path as the default
T277900: [mwcli] golint doesn't appear to be running in blubber for mwcli
T282361: [mwcli mwdd] Removed "static files" can end up being left on disk
T282359: [mwcli] mediawiki: Use .gitreview or composer.json files to see if things exist (like core & Vector)
T282358: [mwcli mwdd] Decide on what to do with the "default" service set and create / destroy commands
T282357: [mwcli mwdd] If mwdd ends up creating a LocalSettings.php file at the request of the user, and Vector exists, load it
T282356: [mwcli mwdd] Figure out how docker-compose overrides can be used
T282340: [mwcli mwdd] Fix mwdd.DefaultForUser().DockerExec output issue (panic: runtime error: slice bounds out of range)
T282350: [mwcli mwdd] composer & phpunit working in different directories
T282360: [mwcli mwdd] Provide a command that can easily tail mediawiki logs
T282331: [mwcli mwdd] fresh integration
T282336: [mwcli mwdd] make use of composer cache from host
T282333: [mwcli mwdd] statsd/graphite integration
T282332: [mwcli mwdd] mysql-replica implementation
T282335: [mwcli mwdd docker-compose raw] Command to run raw docker-compose commands
T282338: [mwcli mwdd] Ensure all key commands have examples in the help output
T282354: [mwcli mwdd] Allow users to choose to end up on ssh or https on gerrit for repos that are cloned
T282365: [mwcli mwdd] Initial documentation
T282363: [mwcli mwdd] Document a debugger workflow

Change 615580 merged by jenkins-bot:

[mediawiki/tools/cli@master] mwdd: Installing with sqlite

https://gerrit.wikimedia.org/r/615580

Change 682240 merged by jenkins-bot:

[mediawiki/tools/cli@master] mwdd: Enable installing with mariadb

https://gerrit.wikimedia.org/r/682240

Change 682310 merged by jenkins-bot:

[mediawiki/tools/cli@master] mwdd: Implement nice destroy commands for individual services

https://gerrit.wikimedia.org/r/682310

Change 682311 merged by jenkins-bot:

[mediawiki/tools/cli@master] mwdd: Implement mediawiki & mysql suspend & resume commands

https://gerrit.wikimedia.org/r/682311