Skip to content

[WIP] Nearest Extreme Point Oracle variant #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

[WIP] Nearest Extreme Point Oracle variant #86

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2d1c16a
intermediate
pokutta Feb 14, 2021
656a9a7
Merge branch 'master' of github.com:ZIB-IOL/FrankWolfe.jl
pokutta Feb 14, 2021
640cd53
working on nep implementation
pokutta Feb 14, 2021
d34d01a
minor -> just moved the check to the right function
pokutta Feb 14, 2021
e463ff9
Merge remote-tracking branch 'origin/master' into nep
pokutta Feb 16, 2021
4a9ad8c
Merge remote-tracking branch 'origin/master' into nep
pokutta Feb 17, 2021
d5c5f21
just update gitignore
pokutta Feb 20, 2021
cca04f1
Merge remote-tracking branch 'origin/master' into nep
pokutta Feb 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions examples/largeScale.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import FrankWolfe
using LinearAlgebra

n = Int(1e9)
k = 1000
n = Int(1e5)
k = 10000

xpi = rand(n);
total = sum(xpi);
Expand All @@ -16,7 +16,7 @@ end
# better for memory consumption as we do coordinate-wise ops

function cf(x, xp)
return norm(x .- xp)^2
return LinearAlgebra.norm(x .- xp)^2
end

function cgrad!(storage, x, xp)
Expand All @@ -33,6 +33,8 @@ FrankWolfe.benchmark_oracles(x -> cf(x, xp), (str, x) -> cgrad!(str, x, xp), lmo
(str, x) -> cgrad!(str, x, xp),
lmo,
x0,
nep=false,
L=2,
max_iteration=k,
line_search=FrankWolfe.agnostic,
print_iter=k / 10,
Expand Down
13 changes: 11 additions & 2 deletions src/FrankWolfe.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ function fw(
verbose=false,
linesearch_tol=1e-7,
emphasis::Emphasis=blas,
nep=false,
gradient=nothing
)
function print_header(data)
Expand Down Expand Up @@ -100,7 +101,7 @@ function fw(
trajData = []
time_start = time_ns()

if (line_search === shortstep || line_search === adaptive) && L == Inf
if (line_search === shortstep || line_search === adaptive || nep === true ) && L == Inf
println("FATAL: Lipschitz constant not set. Prepare to blow up spectacularly.")
end

Expand Down Expand Up @@ -143,7 +144,7 @@ function fw(
@emphasis(emphasis, gradient = (momentum * gradient) + (1 - momentum) * gtemp)
end
first_iter = false

v = compute_extreme_point(lmo, gradient)

# go easy on the memory - only compute if really needed
Expand All @@ -162,6 +163,14 @@ function fw(
(t, primal, primal - dual_gap, dual_gap, (time_ns() - time_start) / 1.0e9),
)
end

# build-in NEP here
if nep === true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if nep === true
if nep

just checking the boolean directly

# argmin_v v^T(1-2y)
# y = x_t - 1/L * (t+1)/2 * gradient
@. gradient = 1 - 2 * (x - 1 / (L * 2 / (t+1)) * gradient)
v = compute_extreme_point(lmo, gradient)
end

if line_search === agnostic
gamma = 2 // (2 + t)
Expand Down