Skip to content
GitLab
Menu
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Gaëtan Caillaut
minibert-oscar
Commits
b92ff5c1
Commit
b92ff5c1
authored
Jan 22, 2021
by
Gaëtan Caillaut
Browse files
Add progress bar in function _split
parent
4d9c3db9
Changes
1
Hide whitespace changes
Inline
Side-by-side
mkdata.py
View file @
b92ff5c1
...
...
@@ -88,7 +88,7 @@ if __name__ == "__main__":
gzip
.
open
(
test_outfile
,
"wt"
,
encoding
=
"UTF-8"
)
as
testfile
,
\
gzip
.
open
(
args
.
input
,
"rt"
,
encoding
=
"UTF-8"
)
as
infile
:
itrain
,
idev
,
itest
=
0
,
0
,
0
for
i
,
l
in
enumerate
(
infile
):
for
i
,
l
in
tqdm
(
enumerate
(
infile
)
,
total
=
n
)
:
outfile
=
None
if
itrain
<
len
(
train_ids
)
and
train_ids
[
itrain
]
==
i
:
itrain
+=
1
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment